<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>Serious Computer Business</title>
    <link>https://octet-stream.net/b/scb/</link>
    <description></description>
    <managingEditor>tom.karpiniec@outlook.com (Thomas Karpiniec)</managingEditor>
    <pubDate>Wed, 05 Aug 2026 15:11:05 +1000</pubDate>
    <item>
      <title>Getting ready for local inference</title>
      <link>https://octet-stream.net/b/scb/2026-06-28-getting-ready-for-local-inference.html</link>
      <description></description>
      <content:encoded><![CDATA[<p>
Recently I’ve been experimenting with local LLMs for coding. Up until a couple of months ago I was a happy GitHub Copilot subscriber. I’d been quite satisfied to let other companies take on the cost of buying and managing hardware, particularly while everything is moving so quickly in the field. Local inference would be <em>ideal</em> for privacy/security/reliability reasons but it’s not a hard requirement for the work I tend to do. Copilot was particularly nice because I could bounce freely between OpenAI’s and Anthropic’s models depending on what felt smarter at the time.
</p>

<p>
It was around then we started seeing big hikes in request cost multipliers, and finally the announcement that Copilot would be <a href="https://github.blog/news-insights/company-news/github-copilot-is-moving-to-usage-based-billing/">moving to usage-based billing</a>. Up to this point I had a distinct feeling that my Copilot bill was oddly good value but I hadn’t realised the degree to which Microsoft had been subsidising the old request-based scheme. The new costs were not <em>unreasonable</em> but they were <em>high</em>, to the point that I was motivated to find a different way to fulfil my AI needs.
</p>

<p>
Please don’t misunderstand me at this juncture—I’m not suggesting that inference isn’t an economically viable service, or that the bubble’s going to pop any day now. It’s simply become clear that I’m not the target customer for the kind of inference that Github Copilot is selling. Today you buy your LLMs through Copilot for the same reason you use Azure. It’s because you’re already up to your neck in Microsoft contracts and it’s easier, or because you need the enterprise data security guarantees, or their filter protecting against accidentally regurgitating copyrighted material. There are plenty of customers for whom it’s not bonkers to pay the Copilot premium. They are not me, and that’s fine.
</p>

<p>
For now, resolving this has been easier than expected. My clients where I’m doing a significant amount of coding are providing/funding AI services for their staff anyway and all I had to do was ask. It turns out, if you think AI is a force multiplier for your employees, extending that to your contractors is a no-brainer.
</p>

<p>
Before that happened, though, I was already in the market for a new work Mac because I needed some higher specs and I’d been watching <a href="https://bsky.app/profile/antirez.bsky.social">antirez posting</a> about his project <a href="https://github.com/antirez/ds4">DwarfStar</a>, in which he gets a pretty serious MoE open weights model running on Macs with 96 or 128 GB of RAM through clever quantisation. I wanted a hedge against token prices going crazy and the best VRAM I had locally was an RTX3060 with 12 GB. Not great. So, I made the call to buy the MacBook with the totally unreasonable 128 GB RAM configuration. Luckily for me, I did so before it <a href="https://www.reuters.com/world/asia-pacific/apple-raises-prices-macbooks-ipads-memory-costs-skyrocket-2026-06-25/">became 33% more expensive the other day</a>.
</p>

<p>
My initial report: I’ve had considerable success using ds4 with <a href="https://opencode.ai">OpenCode</a>, using the q2-q4 imatrix variant of DeepSeek v4 Flash. The model is neither <em>brilliant</em> nor <em>fast</em> but it’s pretty good and pretty reliable at tool calls. I’ve used it for all kinds of tasks from writing Rust, to sysadmin over SSH, to general Q&A. It feels a lot like when I was first using Sonnet 3.5 last year. Now it’s permanently available on my machine at marginal energy cost. That’s quite nice.
</p>

<p>
It really is a wild time. I was a kid in the 90s and 00s when ever-new CPUs were coming out with greater MHz and games changed year on year as video cards (as we called them then) leapt forward in capability. LLMs are the first tech I’ve seen since then where there’s such a rapid pace of development. If I look at my local ds4 setup today, in one sense it’s mildly disappointing. GPT-5.5 and Opus 4.8 run rings around it, especially on the fast and expensive rigs OpenAI and Anthropic use for inference. But if you’d shown this to me 3 years ago I’d have said, “dude, this is fricking incredible!” In this kind of timeline it’s hard to keep perspective.
</p>

<p>
However, life is short. Every second you spend staring at a screen waiting is a second you’re not going to spend again. Nowhere is this felt more keenly than when you’re waiting for a response from your local AI, and you know that Claude would already have finished 30 seconds ago. Oh, the dependability and privacy, but <em>at what cost</em>? I’m neither able nor willing to spend 5 figures on the kind of GPU setup that would let me run a good open-weights model at the context size and token rate offered by the cloud vendors. It’s a bit different from the typical cloud vs on-prem deliberation: yes cloud is better for casual use, but cloud is also better for heavy use. If a situation required it I could justify a beefy Dell PowerEdge; I could not justify the kind of hardware they’re using for these workloads.
</p>

<p>
Until there are breakthroughs in inference performance we have to assume that a good model and context will be relatively slow when run on local equipment. This calls for a different way of working. I really want some kind of framework that I haven’t found yet: something in the spirit of gas town but optimised for running locally where workers are slow and single-threaded. I want to be able to chat interactively about the requirements for my project, report bugs, request features, manipulate what’s being worked on, while leaving it to background processes to actually work on the code, raise PRs or some equivalent, perform quality control, and leave neat work for me to review when I’m ready.
</p>

<p>
This isn’t because I’m eager to relinquish control of my code and go full vibes. It’s because talking about code blow-by-blow with a local AI is slow and not good use of my time on this planet. It’s better if the AI takes on larger chunks of work and I get to interact with it less frequently and more efficiently. I just need some sort of thoughtful scheduling engine and CLI for doing this. I haven’t found it yet. In principle I could try to build it but I don’t want to. There must be thousands of other developers yearning for exactly this thing, and I don’t have a lot of free time. It’s my explicit plan to let somebody else figure this out, then use their tool when it’s done. If you see it then please drop me an email.
</p>

<p>
Until then, ds4 and opencode are pretty neat. If there was an AI or AI-pricing apocalypse tomorrow, I might not be able to work the same way I do with codex/claude, but I have plenty of local capability I could draw on. And even if I'm never forced to rely on it for economic wellbeing, at least I’ll have plenty of RAM for Docker and Electron apps. So far, even without any AI workloads, I’ve reached at least 54 GB RAM in use. Isn’t modern computing grand?
</p>

]]></content:encoded>
      <author>Thomas Karpiniec</author>
      <guid>urn:uuid:934a9890-bd14-475e-a6b8-85d1cbe15579</guid>
      <pubDate>Sun, 28 Jun 2026 21:06:41 +1000</pubDate>
    </item>
    <item>
      <title>The banal failure of 5G home internet</title>
      <link>https://octet-stream.net/b/scb/2026-07-05-the-banal-failure-of-5g-home-internet.html</link>
      <description></description>
      <content:encoded><![CDATA[<p>
Hot take: the viability of home 5G internet in Australia has been destroyed by telcos deploying cheap, inferior modems that will cause most customers to blame cellular connectivity rather than the garbage piece of equipment that Telstra or Vodafone sent them.
</p>

<p>
Back in 2013 Australia had an election and replaced its Labor government with the Liberals. Labor had started building out a National Broadband Network and one of the key parts of this plan was that the vast majority of urban residences and businesses would receive fibre to the premises (FTTP). The Liberals made the case that this was not good use of taxpayer dollars and altered the plan to a “multi-technology mix”, which included pressing into service much of the copper telephone wiring that Telstra had already given up maintaining to provide “fibre to the node” (FTTN). This was a scheme where you still use an ADSL-style modem but you only have to reach your nearest “node” rather than the exchange, so you can get higher speeds.
</p>

<p>
At the time I thought this was complete baloney. I still suspect it was. Politicians: what can you do?
</p>

<p>
However, I was wrong about one thing. I recall that one of the justifications for the multi-technology mix was that 5G services were going to increase and improve and there would be less need for fibre. I didn’t think much of this until a few years ago when, due to short-term living arrangements, I tried out a 5G mobile internet plan with Telstra. Honestly, I was pretty shocked. On a dinky portable modem I was getting solid hundreds of Mb/s down and tens up, at a cost competitive with many of the NBN fibre plans.
</p>

<p>
I live in an urban area with good 5G coverage from multiple providers and I later spent a couple of years using 5G internet seriously at home, even though I had a fibre NBN box sitting there. The NBN box wasn’t in a convenient room so it was nice that I could put the 5G router pretty much anywhere.
</p>

<p>
I started with Vodafone. The modem, a Nokia FastMile 5G, had an intermittent bug where it would stop servicing local DNS requests. The 5G connection was fine—ongoing TCP connections continued normally—but for regular web browsing it was dead. If I didn't power-cycle the router it would eventually recover by itself but this was unpredictable and could take hours.
</p>

<p>
I thought I would log on and change the DHCP settings to point clients at a different DNS server, like 1.1.1.1 or 8.8.8.8. No such luck—the web interface simply didn’t offer a custom DNS server for the DHCP service. This is table stakes configurability and it’s not there.
</p>

<p>
Then, I saw that I could disable the DHCP server entirely on the modem. I thought I would put a Raspberry Pi on the network to act as a replacement that could provide a different DNS server. I discovered that the WiFi/ethernet bridging on the modem doesn’t work properly and while Ethernet-connected peers could be assigned an address by my Pi, WiFi peers would never get one.
</p>

<p>
I parked that plan, and for a while I simply plugged in a second router and double-NATed. The second router was a more-or-less capable TP-Link that allowed me to configure DNS servers. It was inelegant but it worked for a while.
</p>

<p>
Then I had to contend with a different problem with the Nokia: it was frequently downgrading to non-5G connectivity and my speeds would reduce to a few Mb/s. The fix? Slightly rotate the router so it had weaker connectivity to the 5G, then rotate it back again. I'm quite serious. I had to wiggle it on the shelf for about 20 seconds and suddenly it would lock on to 5G again.
</p>

<p>
After a while I decided I was sick of this and I churned to Telstra 5G. I exchanged one set of problems for another.
</p>

<p>
When I had friends over for games we lost connectivity with each other. It couldn’t handle the load of multiple PCs using Steam, whether over WiFi or Ethernet. Around once a week the modem would simply drop its internet connection until I turned it off and on again. This failure mode became consistent enough that <a href="https://tangled.org/octet-stream.net/power-watchdog">I scripted a Raspberry Pi</a> with a WiFi-controlled power outlet to automatically detect when the internet broke and turn the modem off and on again.
</p>

<p>
There are better 5G home routers on the market. The trouble is, you’re not supposed to use them. Both Vodafone and Telstra require that you use exactly the provided device and it would be trivial for them to detect if you were using something else. This is very different from the NBN where you can plug in any Ethernet device that speaks DHCP and get online.
</p>

<p>
Nowadays I am back on the NBN fibre. The performance is good, and thanks to investing in a decent Ubiquiti router I’m no longer plagued by dodgy equipment issues. The fibre is still coming into an inconvenient room and I’ve accepted that I’m going to have to pay a tradie to move it somewhere better. So it goes.
</p>

<p>
By the way, it’s not just me who thinks the modems are worthless. Telstra has a bunch of scary copy saying that when you cancel the plan you must return the modem within N days according to their instructions or they will charge you lots of money. Many months have passed and they have never provided any instructions for how to return it. Not worth their time and money, apparently.
</p>

<p>
The whole thing is profoundly stupid. 5G coverage is quite good and getting better all the time. Infill is continuing so that there are smaller cells and more bandwidth available during peak. There’s a lot to like. You don’t need fibre installed in your building. If you’re a business that needs high availability you can sign up with two different 5G providers that use different cell towers. Yet, the whole thing becomes unworkable simply because the ISPs are cheapskates with their hardware. What a waste.
</p>

]]></content:encoded>
      <author>Thomas Karpiniec</author>
      <guid>urn:uuid:118bbfc8-3a8d-47d5-b4de-a03c5be1a83d</guid>
      <pubDate>Sun, 05 Jul 2026 20:30:54 +1000</pubDate>
    </item>
    <item>
      <title>Hold on to your hats, HaLow is here</title>
      <link>https://octet-stream.net/b/scb/2026-07-22-hold-on-to-your-hats-halow-is-here.html</link>
      <description></description>
      <content:encoded><![CDATA[<p>
I’ve been waiting a long time for <a href="https://en.wikipedia.org/wiki/IEEE_802.11ah">WiFi HaLow (802.11ah)</a> to show up. For the unaware, this is a standardised variant of WiFi using the 900 MHz ISM band to provide range over 1 km with a corresponding lower throughput—up to tens of Mbps.
</p>

<p>
The standard was published in 2017 but for a long time the only available hardware was pretty specialised and I stopped paying attention. Manufacturing has increased and around 18 months ago Heltec started selling a properly convenient USB/ethernet bridge device, the <a href="https://heltec.org/project/ht-hd01/">HT-HD01</a>. Now that I know they exist, I have ordered four of them for mesh experiments. These units use a chip <a href="https://www.morsemicro.com/chips/">designed by Morse Micro</a>, which I am proud to discover is an Australian company<a id="footnote-1-ref" href="#footnote-1">[1]</a>.
</p>

<p>
So what’s the big deal? Well, general consumers have had a pretty annoying range of communication technologies to choose from. We have regular fast WiFi, of course, with its tens-of-metres range. 2.4 GHz goes a bit farther than 5 GHz but competition for channels is fierce and microwave ovens are a natural predator. Bluetooth 5 Low Energy has its wonderful Coded PHY which lets you trade off bandwidth to get up to 1 km range, but it’s really very slow and you’ll be lucky to find an API for it outside of Android. Then there’s <a href="https://en.wikipedia.org/wiki/LoRa">LoRa</a>, which can work up to around 10 km, again with very low throughput. I’ll have more to say about LoRa shortly.
</p>

<p>
We’ve always been missing the happy middle. Now HaLow is here and that’s exactly what it offers. What can we do with it?
</p>

<ul>
<li>Cover large properties with internet access. Or stick an AP on your roof/balcony and enjoy connectivity on your walk.
<li>Offsite backups.
<li>24/7 LAN party with your mate who lives a block away.
<li>Community WiFi projects (this is the one I’m eyeing off).
<li>Probably lots more!
</ul>

<p>
Of course, we’re not strictly limited to what we can buy off the shelf. There’s always amateur radio, the main downside of which is that <a href="https://xkcd.com/191/">you can only talk to other radio amateurs</a>. For community/mesh networking this has some upsides—amateurs are generally well-behaved and the legal requirement to identify stations gives you out-of-band ways to deal with bad actors, which means you don’t have to solve every aspect of abuse handling in the protocol itself.
</p>

<p>
One thing that’s particularly cool about the HT-HD01 is that it supports <a href="https://en.wikipedia.org/wiki/IEEE_802.11s">802.11s</a>, a standard protocol for mesh networking. Here, stations can work together to form an encrypted mesh network. With the appropriate configuration your end devices can see a flat Ethernet segment, as if they were connected to the same switch, and the mesh routing is handled for you behind the scenes. That’s pretty neat.
</p>

<p>
Still, not everybody is satisfied with 802.11s as a multihop router. One of the most popular mesh-routing protocol families is called <a href="https://en.wikipedia.org/wiki/B.A.T.M.A.N.">BATMAN</a>, which is implemented in the Linux kernel as <code>batman-adv</code>. Using something like <a href="https://openmanet.github.io/docs/">OpenMANET</a> it happens that you can use 802.11s for all of its underlying connectivity and encryption but disable just the mesh forwarding, stacking <code>batman-adv</code> on top instead to cover that aspect.
</p>

<p>
I am curious to see how well these could work for a community WiFi deployment. It’s a bit of a harder challenge to get going. Even if the advertised 1–2 km range is realistic you need significant density to get a usable mesh. Within that usable mesh, though, suddenly so much is possible. Not just IRC: actual web pages, forums, email, images, PDFs, even VoIP or games if the latency is good enough. This is the kind of speed where it’s no longer just a poor backup to the “real internet”. If you rig it up in a sufficiently reliable and convenient manner it could be your <em>first</em> choice for local communications, not just something you reach for when Discord stops working.
</p>

<p>
And that’s my main gripe with <a href="https://en.wikipedia.org/wiki/MeshCore">MeshCore</a>/<a href="https://en.wikipedia.org/wiki/Meshtastic">Meshtastic</a>, which are based on LoRa. They work for a low-volume text-based group chat containing only short messages. On the popular MeshCore “Mid” preset a 100-byte packet <a href="https://zindello.com.au/finding-the-sweet-spot-meshcore-lora-settings-in-the-australian-900mhz-band/">takes 0.55 seconds of airtime</a>. That’s fine for what it is but it doesn’t really supplant anything. Good for an emergency or bushwalking; an idle curiosity when life is proceeding normally.
</p>

<p>
My other gripe with LoRa is that it’s a proprietary modulation scheme. The devices may be inexpensive but at the end of the day it’s using a patented and licensed technology. I would prefer to use an open standard where possible.
</p>

<p>
So I am looking forward very much to playing with these devices and seeing what they can do. If you live in Hobart and feel interested in doing some HaLow meshing, drop me an email.
</p>

<hr>

<ol>
<li id="footnote-1">I was moved to take a peek at their <a href="https://www.morsemicro.com/careers/">careers page</a> but, alas, all of their technical roles are on-site up in Sydney. That’s not happening. <a href="#footnote-1-ref">↩︎</a>
</li>
</ol>

]]></content:encoded>
      <author>Thomas Karpiniec</author>
      <guid>urn:uuid:7a28591a-8e7d-4ed9-968e-3a4b9c9997d4</guid>
      <pubDate>Wed, 22 Jul 2026 22:08:54 +1000</pubDate>
    </item>
    <item>
      <title>Automatic JavaVM and Context detection using Rust on Android</title>
      <link>https://octet-stream.net/b/scb/2026-07-23-automatic-javavm-and-context-detection-using-rust-on-android.html</link>
      <description></description>
      <content:encoded><![CDATA[<p>
When you’re running Rust code in the context of an Android app, if you want to work with Android-specific functionality over JNI you tend to need three things:
</p>

<ol>
<li>A reference to the running Java virtual machine.
<li>A reference to a suitable <code>Context</code>, often the <code>ApplicationContext</code>.
<li>Some helper code on the Java side to provide APIs with required callbacks, and so on.
</ol>

<p>
My cross-platform crate <a href="https://crates.io/crates/netwatcher">netwatcher</a> has to do all of these when it finds itself running on an Android. It handles items 1 and 2 in a very straightforward manner: it exposes <a href="https://github.com/thombles/netwatcher/blob/6ee4b450671853392a258cff8639086cd4e3b925/src/android.rs#L18-L51">a public function</a> where you pass in both of them. It’s up to the developer to do the appropriate plumbing to call through from Java or Kotlin, passing along their <code>Context</code> object. For item 3, I was happy to discover (that is, copy from someone else) a fiddly-but-dependable technique involving injecting DEX bytecode at runtime, <a href="/b/scb/2025-08-03-injecting-java-from-native-libraries-on-android.html">which I wrote about previously</a>.
</p>

<p>
As a crate developer, the holy grail is to get all of these runtime dependencies automatically without the end user having to do anything other than put the crate in <code>Cargo.toml</code> and use the API normally. They will be sad if their code suddenly breaks weirdly when they cross-compile it to Android. The difficulty level is increased by being a mere dependency within a larger Rust binary—if we’re just one piece of the final <code>.so</code> then we can’t/shouldn’t hook into <code>JNI_OnLoad</code> (without coordination through the final library), which is otherwise a reliable way to get the <code>JavaVM*</code> pointer.
</p>

<p>
The other day I was following some links between issues on GitHub and I noticed that the Matrix Rust SDK has apparently solved items 1 and 2, making them entirely automatic (<a href="https://github.com/matrix-org/matrix-rust-sdk/pull/6306">PR #1</a>, <a href="https://github.com/matrix-org/matrix-rust-sdk/pull/6370">PR #2</a>). I was intrigued and went on a deep dive to see how this worked. What I found is… well it’s pretty out there. For now I’ve decided to leave netwatcher the way it is, even if it’s more annoying for the developer.
</p>

<p>
The hacks involved are pretty impressive so here I want to dissect how they work and briefly discuss the risks. To be clear, I’m not trying to put anyone on blast for writing or using these techniques. It’s a noble cause and if you get it wrong it’s probably just going to crash.
</p>

<p>
Let’s begin with item 2 since it’s more straightforward. There is a class in Android called <code>ActivityThread</code> which you won’t find in the documentation. It has a static method <code>currentActivityThread()</code>, and when you get the instance back you can call <code>getApplication()</code>. This returns the <code>Application</code> that you can use as a <code>Context</code>. JNI can give you a reference to the class and help you call the methods. Too easy.
</p>

<p>
<a href="https://github.com/matrix-org/matrix-rust-sdk/blob/c49ad735c21f81573adca020ad0d8e302bd657f8/bindings/matrix-sdk-ffi/src/platform/android_platform.rs#L64-L77">From the Matrix Rust PR</a>:
</p>

<pre style="background-color:#f8f8f8;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic">// Get the current activity thread
</span></span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"></span><span style="color:#204a87;font-weight:bold">let</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">activity_thread</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#ce5c00;font-weight:bold">=</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">env</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline">    </span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">call_static_method</span><span style="color:#000;font-weight:bold">(</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline">        </span><span style="color:#4e9a06">&#34;android/app/ActivityThread&#34;</span><span style="color:#000;font-weight:bold">,</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline">        </span><span style="color:#4e9a06">&#34;currentActivityThread&#34;</span><span style="color:#000;font-weight:bold">,</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline">        </span><span style="color:#4e9a06">&#34;()Landroid/app/ActivityThread;&#34;</span><span style="color:#000;font-weight:bold">,</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline">        </span><span style="color:#ce5c00;font-weight:bold">&amp;</span><span style="color:#000;font-weight:bold">[],</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline">    </span><span style="color:#000;font-weight:bold">)</span><span style="color:#ce5c00;font-weight:bold">?</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline">    </span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">l</span><span style="color:#000;font-weight:bold">()</span><span style="color:#ce5c00;font-weight:bold">?</span><span style="color:#000;font-weight:bold">;</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline"></span><span style="color:#8f5902;font-style:italic">// Then get the application context
</span></span></span><span style="display:flex;"><span><span style="color:#8f5902;font-style:italic"></span><span style="color:#204a87;font-weight:bold">let</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">context</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#ce5c00;font-weight:bold">=</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#000">env</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline">    </span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">call_method</span><span style="color:#000;font-weight:bold">(</span><span style="color:#000">activity_thread</span><span style="color:#000;font-weight:bold">,</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#4e9a06">&#34;getApplication&#34;</span><span style="color:#000;font-weight:bold">,</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#4e9a06">&#34;()Landroid/app/Application;&#34;</span><span style="color:#000;font-weight:bold">,</span><span style="color:#f8f8f8;text-decoration:underline"> </span><span style="color:#ce5c00;font-weight:bold">&amp;</span><span style="color:#000;font-weight:bold">[])</span><span style="color:#ce5c00;font-weight:bold">?</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span><span style="display:flex;"><span><span style="color:#f8f8f8;text-decoration:underline">    </span><span style="color:#000;font-weight:bold">.</span><span style="color:#000">l</span><span style="color:#000;font-weight:bold">()</span><span style="color:#ce5c00;font-weight:bold">?</span><span style="color:#000;font-weight:bold">;</span><span style="color:#f8f8f8;text-decoration:underline">
</span></span></span></code></pre>

<p>
The problem is that <code>ActivityThread</code> and its methods aren’t officially part of the Android SDK and there are <a href="https://developer.android.com/guide/app-compatibility/restrictions-non-sdk-interfaces">policies around using non-SDK interfaces</a>. When I look up <code>currentActivityThread()</code> and <code>getApplication()</code> in their spreadsheet they are both marked <code>unsupported</code>. This means Android turns a blind eye: the call will be permitted, but they’re allowed to break how this works at any time for any reason.
</p>

<p>
Going by <a href="https://github.com/search?q=Landroid%2Fapp%2FActivityThread&amp;type=code">a quick GitHub search</a> this technique gets a fair amount of use but it still gives me pause. I don’t want to suddenly have to rearchitect my library because a new version of Android changes how <code>ActivityThread</code> works, or handle bug reports from people who are running strange OEM Android distributions where up is down and down is up. For now I intend to stick with the supported API.
</p>

<p>
For item 1 we must turn to the crate <a href="https://crates.io/crates/jvm-getter">jvm-getter</a>, which is where Matrix has ended up. This crate helpfully begins its README with a warning icon and suggests alternatives.
</p>

<p>
The game here is to locate and call a function called <code>JNI_GetCreatedJavaVMs</code>. This is <a href="https://docs.oracle.com/en/java/javase/11/docs/specs/jni/invocation.html#jni_getcreatedjavavms">an official JNI invocation function</a> that gives you a list of <code>JavaVM*</code> pointers. In an Android app you would expect there to be just one VM, so you can pass it a 1-length array and now you have the pointer you need. This function is relatively unusual in JNI because it’s free-standing, an exported symbol you can look up. For most JNI work you would instead start with a <code>JavaVM*</code> or a <code>JNIEnv*</code> and use its function table.
</p>

<p>
This function has always been exported. What has changed over time is whether it can be discovered by application NDK code. Up to API level 23, you could use <code>dlopen</code> and <code>dlsym</code> to look it up by name and then call it. In API levels 24–30 linker namespaces were introduced and <code>JNI_GetCreatedJavaVMs</code> was no longer accessible, so <code>dlsym</code> fails. Then from API level 31 onward, Android made <code>libnativehelper.so</code> a public NDK library and this JNI function became available to <code>dlsym</code> officially.
</p>

<p>
This quirk bit Matrix: originally they used libloading (a cross-platform wrapper around the <code>dlopen</code>/<code>dlsym</code> technique) to look up the symbol and call it (PR #1). A <a href="https://github.com/matrix-org/matrix-rust-sdk/issues/6368">bug report came in</a> that it crashes on Android 11 and they switched to using the <code>jvm-getter</code> crate, which offered a workaround when <code>dlsym</code> fails.
</p>

<p>
So, that workaround. Buckle up, kids. <a href="https://github.com/gobley/jvm-getter/blob/main/jvm-getter/src/android.rs">What it does</a> is this:
</p>

<ol>
<li>Use a system property to learn the name of the Android runtime library, typically <code>libart.so</code>.
<li>Use <a href="https://www.man7.org/linux/man-pages/man3/dl_iterate_phdr.3.html"><code>dl_iterate_phdr</code></a> to walk all the shared objects that have been loaded into the current process. For each one, this invokes a callback.
<li>In the callback, look for the one corresponding to <code>libart.so</code>. When you see it, record the base address at which it’s been loaded into memory, and the full path on disk to the ELF file.
<li>Read the contents of the ELF file into memory.
<li>Use the <a href="https://crates.io/crates/goblin"><code>goblin</code></a> crate to parse the ELF file, find the symbol table, and locate the symbol whose name matches <code>JNI_GetCreatedJavaVMs</code>.
<li>Take that symbol’s relative address.
<li>Add the symbol’s relative address to the base address of <code>libart.so</code> which was recorded earlier.
<li>Cast this to a function pointer and return it to the caller, which then calls it to find the JavaVM.
</ol>

<p>
Now <em>that’s</em> persistence. Notice that the linker namespaces aren’t an isolation boundary. If you can find the address, you can still call it.
</p>

<p>
While this is plainly nuts, on balance it doesn’t seem too bad. There’s nothing wrong with locating <code>JNI_GetCreatedJavaVMs</code> with <code>dlopen</code> and <code>dlsym</code>—Android explicitly makes it available to find on most modern versions and this will be a reliable method for item 1 going forward. The tricky part is the gap in API levels where the symbol isn’t readily available, if you don’t want to drop support for older devices yet.
</p>

<p>
The thing about the ELF-parsing workaround is that all the relevant Android versions are <em>in the past</em>. They’re old phones or tablets that aren’t getting updates. <code>libart.so</code> isn’t going to wake up one day with stripped symbols<a id="footnote-1-ref" href="#footnote-1">[1]</a>. If this workaround turns out to work well across a range of device types then it’s probably going to <em>keep</em> working.
</p>

<p>
Unfortunately, solving item 1 alone isn’t enough to simplify my public API. If I’m not prepared to use an unsupported method to get the <code>Context</code> then I need a configuration method anyway, and it’s just as easy to grab the <code>JavaVM*</code> at the same time.
</p>

<p>
We just need a slightly better way to get an application context from the NDK and everything will be perfect…
</p>

<hr>

<ol>
<li id="footnote-1">I suspect <code>jvm-getter</code> could be made more resilient here by searching <a href="https://docs.rs/goblin/0.10.7/goblin/elf/struct.Elf.html#structfield.dynsyms"><code>dynsyms</code></a> instead of <code>syms</code>. Since the <code>JNI_GetCreatedJavaVMs</code> implementation has to be exported, it must be named and cannot be stripped from this particular symbol table. <a href="#footnote-1-ref">↩︎</a>
</li>
</ol>

]]></content:encoded>
      <author>Thomas Karpiniec</author>
      <guid>urn:uuid:9b092c22-3002-43fc-a551-d39b12c6de8a</guid>
      <pubDate>Thu, 23 Jul 2026 21:35:22 +1000</pubDate>
    </item>
    <item>
      <title>Using local DeepSeek v4 Flash in Xcode</title>
      <link>https://octet-stream.net/b/scb/2026-08-05-using-local-deepseek-v4-flash-in-xcode.html</link>
      <description></description>
      <content:encoded><![CDATA[<p>
I’m rather pleased. I’ve been able to bolt together enough pieces of software that I can now run DeepSeek v4 Flash with native MXFP4 routed-expert weights on my MacBook Pro, not only via a CLI harness like OpenCode, but also through Xcode’s native “intelligence” integration.
</p>

<p>
Before you get too excited please realise that I’m running this on a ludicrously expensive M5 Max with 128 GB RAM. <em>However</em>, I am hoping that the upcoming Qwen3.8-27B release will be awesome and it will be practical to have a similar workflow with a considerably more modest unified RAM budget.
</p>

<p>
This is the stack I’ve ended up with:
</p>

<ul>
<li>Xcode Intelligence configured with a custom ACP agent
<li>OpenCode running as <code>opencode acp</code>
<li>antirez’s <a href="https://github.com/antirez/ds4">ds4-server</a>
<li>DeepSeek v4 Flash: the new 0731 update<a id="footnote-1-ref" href="#footnote-1">[1]</a>
</ul>

<p>
To prove it works I typed <a href="https://gist.github.com/thombles/32d21ec7c3ac1773a89234ed8cee28c7">a short prompt</a> into Xcode—I wanted a Mac SwiftUI application to help me practise cardinal numbers in Polish. It whirred away autonomously for about 90 minutes, filling the context up to 95,200 tokens, and produced an app that more or less did what I asked. I spent an extra half an hour tinkering interactively through the chat interface to fix a few specification issues<a id="footnote-2-ref" href="#footnote-2">[2]</a> and a couple of layout bugs<a id="footnote-3-ref" href="#footnote-3">[3]</a>. The session started at around 17 output tokens/sec and dropped to about 15.
</p>

<a href="https://octet-stream.net/assets/serious/ds4fxcode/app-1-menu.png"><img src="https://octet-stream.net/assets/serious/ds4fxcode/app-1-menu.png"></a>

<a href="https://octet-stream.net/assets/serious/ds4fxcode/app-2-questions.png"><img src="https://octet-stream.net/assets/serious/ds4fxcode/app-2-questions.png"></a>

<a href="https://octet-stream.net/assets/serious/ds4fxcode/app-3-feedback.png"><img src="https://octet-stream.net/assets/serious/ds4fxcode/app-3-feedback.png"></a>

<p>
For a fun twist, I made the app itself <em>also</em> use the ds4 chat completions endpoint to provide some dynamic feedback on what the user typed, particularly to explain any mistakes. (As you can see, the “system prompt” needs improving.) DS4F happens to be weirdly good at handling broken Polish and explaining grammar. I’ll have more to say about that in a future post. I hope it got it right in the code. Otherwise I’m going to be practising the wrong thing.
</p>

<p>
I have no strong opinions about the Swift code. From a quick skim it all seems pretty sensible and it made proper use of <code>JSONSerialization</code> and <code>URLSession</code> for making the HTTP request to the completions endpoint.
</p>

<p>
Assuming you’d like to try something like this yourself<a id="footnote-4-ref" href="#footnote-4">[4]</a>, here are some notes and observations.
</p>

<p>
Let’s start at the bottom of the stack. Since I began my experiments antirez has now merged all the relevant work so you should be able to just clone the ds4 repo, run <code>make -j</code> and run <code>ds4-server</code>. I’m executing it like this:
</p>

<pre>
"$HOME/ds4/ds4-server" \
    --ctx 393216 \
    --kv-disk-dir "$HOME/.ds4/server-kv" \
    --kv-disk-space-mb 100000 \
    --host 0.0.0.0 \
    --ssd-streaming \
    -m "$HOME/ds4/gguf/DeepSeek-V4-Flash-MXFP4Experts-F16HC-F16Compressor-F16Indexer-Q8Attn-Q8Shared-Q8Out-chat-v2-mxfp4-0731.gguf"
</pre>

<p>
Most early benchmarks indicate that DS4F works best with “maximum” thinking. DeepSeek recommends only using max thinking with a 384K context size, which is enforced in <code>ds4-server</code>. Unless you choose a context of at least this size you’ll automatically get downgraded to high thinking. This comes with a RAM cost but it’s mild next to the weights themselves.
</p>

<p>
You’ll want to enable SSD streaming unless you have a Mac with a stupendous amount of RAM. Note that so long as you’re not going into heavy swapping this will cause large amounts of disk <em>reads</em> rather than writes so it shouldn’t contribute unduly to SSD wear. To give you an idea, across this coding session <code>ds4-server</code> has read about 22 TB from disk.
</p>

<p>
Since I have free disk space and generating the tokens is relatively expensive I choose to have a fairly large on-disk KV checkpoint cache. 100 GB is probably overkill for most purposes.
</p>

<p>
If you’re only using it locally you probably won’t want to bind to <code>0.0.0.0</code> but I sometimes access it over Tailscale so an endpoint available over LAN often suits me.
</p>

<p>
So you run that and now you have a standard OpenAI-like chat completions endpoint running at <code>http://127.0.0.1:8000/v1</code>.
</p>

<p>
Next we need <a href="https://opencode.ai">OpenCode</a>, which I have installed through Homebrew. It includes an implementation of <a href="https://agentclientprotocol.com/get-started/introduction">Agent Client Protocol</a>, which is what Xcode needs.
</p>

<p>
Now, in my OpenCode config my model is defined like this:
</p>

<pre>
      "models": {
        "deepseek-v4-flash": {
          "name": "DeepSeek V4 Flash (ds4.c local)",
          "reasoning": true,
          "limit": {
            "context": 393216,
            "input": 393216,
            "output": 384000
          },
          "options": {
            "reasoningEffort": "max"
          },
          "variants": {
            "low": {
              "disabled": true
            },
            "medium": {
              "disabled": true
            },
            "high": {
              "disabled": true
            },
            "max": {
              "reasoningEffort": "max"
            }
          }
        }
      }
</pre>

<p>
And at the top level I have:
</p>

<pre>
  "compaction": {
    "reserved": 65536
  },
</pre>

<p>
This is doing a few things. It forces DS4 to always use max effort even if ACP might try to default to a weaker level. It tells OpenCode the context size available. It allows large responses, which is important when using max thinking because the default limit is 32K and this model will quite happily think for more than that at a time. The reservation gives more breathing space for compaction. Having a large output limit can mess up OpenCode’s calculation of the compaction point, so it will constantly be compacting at a small context size. I fixed that by setting a large input limit too.
</p>

<p>
Note that you also need an environment variable <code>OPENCODE_EXPERIMENTAL_OUTPUT_TOKEN_MAX=384000</code> in Xcode's settings, and in your shell if you use OpenCode there.
</p>

<p>
While you need to have <code>ds4-server</code> running yourself, Xcode will launch OpenCode on its own when you start a conversation. You need to configure it through Xcode’s Settings in the Intelligence section. Add <code>opencode</code> as the path, and scroll down to set <code>acp</code> as the argument and the above environment variable.
</p>

<a href="https://octet-stream.net/assets/serious/ds4fxcode/acp-1.png"><img src="https://octet-stream.net/assets/serious/ds4fxcode/acp-1.png" style="width:50%"></a>

<a href="https://octet-stream.net/assets/serious/ds4fxcode/acp-2.png"><img src="https://octet-stream.net/assets/serious/ds4fxcode/acp-2.png" style="width:50%"></a>

<p>
Like me, you may be confused by how you actually talk to it, since Xcode defaults to Codex. Once you click the Coding Assistant button in the top left, you then need to click on the little “New message” icon to get a dropdown where you can choose the agent you just added. When you type in a message and hit enter it will start OpenCode in the background. After 20–30 seconds of I’m-not-sure-what-exactly, requests start hitting DS4.
</p>

<a href="https://octet-stream.net/assets/serious/ds4fxcode/new-menu.png"><img src="https://octet-stream.net/assets/serious/ds4fxcode/new-menu.png" style="width:30%"></a>

<p>
For some reason my Xcode uses a lot of CPU while DS4 is thinking. Maybe it’s constantly re-rendering the thinking box as reasoning streams in? If I swap away from the coding assistant back to the project view then it settles down. It’s clear that the Xcode UI designers were not anticipating DS4 quantities of thinking—when I click on a large one it can take a full couple of seconds for the scrollable popup to appear.
</p>

<p>
If you get all that right, you should be off to the races. I’m looking forward to trying this out with the new Qwen model too. It’s unclear to me whether Xcode will make it easy to select multiple open models through OpenCode itself or whether I’ll need to set up wrapper scripts and create two ACP entries if I want to toggle between them.
</p>

<hr>

<ol>
<li id="footnote-1">Apparently so called because it was released on July 31. <a href="#footnote-1-ref">↩︎</a>
</li>
<li id="footnote-2">Unsurprisingly in hindsight, it turns out that if you select random numbers between 1 and 999,999, most of the time you’re writing out 6-digit numbers like 556,195. This feels pretty laborious. Most of the grammatical trickiness a student of Polish needs to practise can be reproduced with smaller numbers. <a href="#footnote-2-ref">↩︎</a>
</li>
<li id="footnote-3">DeepSeek v4 Flash only has text inputs so I can’t just feed it a screenshot. It would be an interesting follow-up to see if I can get it to use a textual description of where the components are laid out in the actual window and use that as feedback to determine whether it wrote the SwiftUI it intended. <a href="#footnote-3-ref">↩︎</a>
</li>
<li id="footnote-4">Be aware that DeepSeek v4 Flash cloud inference is extremely cheap. That’s a much more cost-effective way to get tokens than buying hardware with lots of RAM. However, there will always be some people who are interested in local inference, for various reasons. You know who you are. <a href="#footnote-4-ref">↩︎</a>
</li>
</ol>

]]></content:encoded>
      <author>Thomas Karpiniec</author>
      <guid>urn:uuid:e1e2225e-3ff3-46f0-a898-1d48d1d33eb7</guid>
      <pubDate>Wed, 05 Aug 2026 15:07:20 +1000</pubDate>
    </item>
  </channel>
</rss>