While I was watching some Level1Techs videos Wendell I found out about Devember TLDR: Every day in November you tinker with something for an hour or so and share what you did.

Well this is my attempt at that. I say attempt because many,many, many of my previous endevours have ended the same way where I start and then just jump off the train and never catch up.

Day: 0

While I had some ideas what to do, I got alot done (granted it was more than an hour and its a Sunday so dont expect same levels of output every day). What got done was a NAS Buddy backup, setting up an Avalon instance and cleanup of my ansible repository.

NAS buddy backup

This was a long time idea that I wanted to execute and on the second try we actually did it. Not much to say, abit of googling, abit of LLM debugging and its done.

I am dissapointed that you need sudo to run ZFS tooling and there is no good way to restrict access. I dont even trust myself to run these things properly, so having safeguards in place is a good thing.

Avalon

While ollama.cpp itself is great for running models on whatever hardware you have, the specialization you can get with some of the forks is awesome, you do have to be careful tho! You are still running random code from the internet!

That being said I was running Strata but that project started focusing on becoming a harness rather than an engine and intertwined them in a way that just does not make too much sense. It is hard to measure how well the model does it job so tokers per second is the number that must go up because bigger is better!

I was trying to do some things to speed up using the stock llama.cpp but then I found out about Avalon. Tried it and it works pretty well, I do have to tinker with the settings but I dont want to fall back into the tool optimization hole. Im sure I can get more performance out of my setup but for now it works.

Ansible

Letting the LLM take the wheel for your things is fun (because things get done) but you will walk into a landmine soon. I havent blown anything up that I have not wanted (yet). But to counteract that I took the time to clean up the repository and let the LLM explain to me what is going on.

Not too much to talk about this, just computer janitor stuff. The janitor must know where things are!

Day: Next

I do not have a plan of what im going to do, what is the focus of the month. This is something I will have to think through the next day or so. Specially with LLMs it is very easy to go fast … in a circle.

The only thing I know what I want to do is create something but it cannot be tooling. Im way too comfortable in just falling into the tooling part of any project and endlessly tinker with it to create the best environement ever without actually making any progress.

NOTE: This post techincally is written and published on the 2nd of November but its 01:30 and I havent gone to sleep yet so its still the first in my heart!