I just want to talk about failure for a bit. One of the best parts of working with Embedded systems (Arduino and PlatformIO for me) is when you get things right. Somehow all of the fails make it that much more satisfying when things do work.

Example: AI programming

I have been using AI programming heavily for more than two years now. Coding assistants such as GitHub Co-Pilot, Cursor IDE, Cline, Roo Code, gemini-cli, claude-code, opencode and aider (and many more!). All of these have their own way of doing things and take time to set up correctly. When things fail to work you always have to check: is this a failure in my understanding or the tool itself?

The most recent issue I have been addressing with various tools is “Context overrun” – basically the AI coding assistant has a limited amount of tokens or memory which it can use to do it’s text prediction (writing code). Once you fill the context up, the coding assistant needs to either drop some of the text (code) from the “context window” or the response from the associated LLM will be an error.

Some of the assistants have tools to deal with this – which can be simply dropping important files or compressing (summarising) the full text, OR just informing the user (me) that we have an issue. There is no good way to deal with this. We can lose important information either way.

SmartPoi-JS-Utilities

In it’s current form, SmartPoi-js-utilities is mostly a very large html/css/js file which takes up 30 000 tokens of an LLM context window. To put that into perspective, DeepSeek has a Total context window or 65 000 tokens, so just one file is using up half.

I had to re-factor (split the file up) in order to be able to make changes with AI without running out of context. I tried claude, gemini and opencode but none of these worked. They all ran out of context while trying to keep all old and new files in memory simultaneously.

Aider-ce is the best?

Aider is my favourite AI coding assistant. Recently, developments in AI coding such as MCP and LSP meant that Aider was falling behind. So the community came together and added those to a forked version (which is currently still compatible with original aider version): aider-ce (Aider Community Experimentation). This version has a very sophisticated method of tool calls which enable very fine grained edits, something not seen in other AI editing packages – as well as MCP integration and access to built-in language packs (rules about how computer languages are supposed to work.)

In addition to the upgrades mentioned above, the new version incorporates a “todo” system which keeps track of progress (adding Agentic behaviour previously lacking in Aider) – sometimes you want the AI to just “get on with it” and complete a bunch of steps – like re-factoring 30 000 token file!

It’s not perfect, a work in progress actually, but with this new /navigator mode fine-tuned editing capabilities in combination with Aider’s already amazing multi-file editing with git integration I think I have the tools to do almost any job. Certainly we are getting much closer to the ultimate tool for AI assisted coding – for people who care about:

1. Open Source
2. User Control (you can check every step, /undo at any point)
3. Choice (choose MCP service, whether to edit whole or just tiny parts of the file, coose LLM)

Thanks to the Devs at Aider and Aider-CE

I can’t name everyone (Dustin Washington is the maintainer of aider-ce, original Aider created and maintained by Paul Gauthier), but their handles are in the commit messages attached to each painstaking line of code. Making my code almost effortless. It’s not about the code after all, it’s about getting a job done. Teamwork and never giving up.

Conclusion:

I should have called this blog post “A peek under the hood” because that is what we are doing here. At the end of the day the users of SmartPoi don’t care HOW the code is produced, just that it works. If I did it right, the fact that I totally re-factored the entire codebase of SmartPoi-js-utilities should be completely impossible to see from the front end. There is effectively no difference, all it means is that it is now easier for me to work on the code.

The point is that I spent a week doing this so we can move forward with better things – and by the way I did the same for the Magic Poi website. I guess you could say that I just suck at JavaScript.. after all, the Python back-end is just perfect, as-is :-p

Notes:

Even with Aider a full re-factor is not smooth sailing. I used a clever trick to fix the problems after completing the upgrade (problems like missing functions, incorrect addressing..)

git diff main -- Combined_APP/main.js > gitdiff.txt

Then just get the AI to reference the diff and search for any missing functions – or if any errors come up at least we have a working code reference point.

I really must look at the possibility of adding this as a tool to aider-ce. Git is already integrated, so it might just be a simple job to include diff checking for re-factor, which after all is a very common thing for developers to have to do.

The problem:

For the Magic Poi project, the online website and api for the IOT devices to connect to (Poi) is all done with Flask. Flask handles the web front and back-end using Jinja, a templating library. In Flask you have templates with placeholder variables like {% image1 %} which will put “image1” into the web page for example, where you need it.

In traditional JavaScript we would have separate .js .css and .html files – and the JavaScript can be separated into pieces – with each part handling a particular section. For example in Magic Poi we have Images, Timelines, Parties, Friends and more. Using Jinja, we can do all of the heavy lifting (and syncing with poi) in Python on the server, and the placeholders make the html simple to tie in to the rest of the code (we just use the same names and logic) but that comes with a limitation: we cannot have a separate .js and .css file if we want to use these placeholders, it’s an unfortunate limitation of Jinja templating.

So that left me with the main “profile” page ending up with thousands of lines of html, css, and js after adding all the features up until now (and we are not nearly done).

Time to re-factor

Re-factoring is the process of moving code around to make it either easier to read or easier to extend and update – or in this case, both. Having more focused sections would make it easier for me to follow the code, but how would having a bunch of smaller files make it easier to extend? Well it turns out that Aider, my preferred AI assistant until recently, has a problem. In order to edit files it needs to send them through to the LLM (in my case, mostly DeepSeek). It sends the whole file – and profile page was up to 30 000 tokens (token, syllable or part of a word). DeepSeek has a maximum of 65 000 tokens, including the response, so I would often run out of tokens just trying to ask about some change I wanted made in the functionality.

Luckily I found a solution, but it wasn’t easy!

First of all, since the Aider tool cannot edit for me and I really don’t have the time to look at thousands of lines of code (multiple JS Classes, with tens of functions each) I turned to another tool – gemini. Gemini is not as capable as DeepSeek, however the official cli tool is free, and most importantly it has a context limit of 1 million tokens. Since the re-factoring was a basic operation I used gemini to do it – after many hours it was done with a web page that didn’t quite work. Like most of AI coding it was just close enough though…

I guess it might help to describe what I was trying to do. Simply put, I needed to instead of having one “profile.html”, have multiple .html files. So the back-end renders profile.html and it has sections like this:

<div id="tab-content-wrapper">
        {% include 'profile_timelines_section.html' %}
        {% include 'profile_my_images_section.html' %}
        {% include 'profile_shared_images_section.html' %}
        {% include 'profile_friends_section.html' %}
        {% include 'profile_parties_section.html' %}
        {% include 'profile_my_poi_section.html' %}
    </div>

We also have .html files included with associated JavaScript (and css). That way, the timelines section consists of 3 files for example (it’s a little bit more complicated like I have some shared parts also but mainly 3 sections for each functional piece). The main thing is that these are a lot shorter than the original profile.html.

Aider wasn’t cutting it

I had recently been looking at the “Community Edition” of Aider. This differentiates itself with the ability to incorporate “MCP” servers (add-ons) which for example can search the web, control the browser for testing, check code documentation and more. A new option in the Aider-CE is the “Navigator” mode which uses search and replace within files instead of the default “send the whole file” functionality.

I used aider-ce to fix up the issues introduced by gemini. Mostly missing or duplicate code. Easy to fix if you just compare the original with the new version.

Conclusion

The website, https://magicpoi.com has the new re-factored code up and running – and after a week of working on re-factoring it now looks and acts exactly the same as before.

I learned that Aider has some serious limitations, but that is being looked at by the community of programmers who use it (and tested by people like me).

And I learned about a new pattern for my Flask/Jinja templates, which really works.

Going forward

I am now using aider-ce instead of main aider branch.

But – I found another open source coding assistant which seems to be built from the ground up with the limitations of LLM’s like DeepSeek in mind. OpenCode. This project has many bells and whistles like MCP extensibility, a kind of “Architect” mode (similar to Aider but it keeps a list of “todo’s” and executes them in order) and more.

Stay tuned for more about that!