I have spent the better part of the past month researching Agentic AI. I was inspired by Andrej Karpathy’s LLM Wiki concept which immediately suggested to me a helpful to tool to more rapidly bring myself up to speed on all things Universal Basic Income.
Recommendation: It is well worth your time and a modest cash investment to play with Agentic tools. At the outset minimize the number of variables at play by experimenting via Claude Code and Pro subscription ($20/month). Your goal should be understanding how to program LLM Agents to meet your needs. Start simple, work your way up. It is no exaggeration to say the results are magical.
The Wiki LLM Concept
The basis of Karpathy’s insight, which may also have predecessors before his blog post, was that LLM are good at taking inputs and summarizing them and identifying connections between topics. Effectively building a knowledge wiki – and that in point of fact the hardest part of building a wiki is maintaining a wiki.
His model has three core components:
- Source files: These are human identified and selected and represent a source of “truth.” Garbage in – garbage out; careful curation is required.
- Wiki: The LLM will be tasked with ingesting the source files and building (updating) the wiki with new concepts and summarizes and linking connections.
- Agent Instructions: These instructions provide the LLM with a persona, guidance for how to ingest new files, guidance for how to respond to queries, and guidance on how to periodically clean the wiki.
Rabbit Holes
I pursued a number of concepts which by and large did not pan out or where beyond my capabilities.
- Open Source: Among many other things I have increased my tempo for exploring all things open source – primarily in the interests of privacy. This included installing a dual boot with Linux Mint on my laptop and searching for software solutions that minimized the leakage of my data to digital corporations. Mind you my starting point left lots of room for improvement. The point is that there is a lot of good open-source alternatives but that they really require committing to climbing the learning curve – and when multiple tools are involved it gets overwhelming in a hurry.
- Local AI Models: In many ways this was the most straightforward issue to address. Using Ollama and their model library it is possible to download and run AI locally – even on my very modest and moderated dated laptop. A couple of key points: without a GPU the range of LLM models available will be quite limited and the quality and speed of output modest. One can select a model with less precision that trades off the size and speed of the model against quality of the outputs – a quanticized model. The second factor to consider is that the model must support agentic tools and reasoning in order for it to function with Agents. It goes without saying that it is worth the effort to download and test various models to see how they perform relative your own needs.
- Hardware Requirements: Fundamentally a key tradeoff is how much are you willing to pay (money and effort) to maintain local AI relatively to paying. Somethings I learned that maybe useful: GPUs and in particular VRAM is king – more VRAM allows for the loading of larger, and better, LLM models. For intel based machines (Windows or Linux) this will require a dedicated GPU card – these get expensive in a hurry. Apple machines have integrated RAM which bypasses the need for an explicit GPU card – which is why so far as I can tell so many AI hobbyists seem to have Apple products – albeit with huge RAM capacity 32GM and up. MacMini seems to be a favorite here. Where I am today in my AI journey – I’m not quite willing to shell out a couple grand on new hardware – so this was out.
- Local Agentic Tools: There are many agentic tools available – but two that seemed to be constantly at the forefront are Claude Code and OpenClaw. The former is proprietary, fee based, and focused on ease of use and security whereas the latter is open source and focused on maximizing the functionality of the tool. OpenClaw more clearly allows the use of local AI models.
- Security: This was the issue that tipped it for me: OpenClaw is a very powerful, by design and by intent, tool. But every, and I mean every, reviewer said to run it on a dedicated machine without access to any personal mission critical information (passwords, financial, emails, etc) as in theory it has very broad-based authorizations to edit files and take actions. Not having sufficient experience or technical knowledge I thought this was too big a risk and that the mitigation buying a whole new computer set up seemed unjustified given my current state of learning.
First LLM Wiki
Preliminary test with a few articles shows a lot of promise. The LLM was able to ingest each article, summarize it, extract key concepts, and link these elements together. It was then able to answer a question based on the underlying wiki. Pretty impressive really. Iterative engagement with the AI including identifying new issues and search for relevant scholarly articles lead to much greater insight on a rapidly accelerated timeline.
- Identify an initial set of articles around a question. Upload and ingest. The first time this happens is truly magical.
- Ask questions of the LLM wiki and identify gaps in knowledge – search out other articles to plug those gaps. I found it helpful to include pro/con positions so that I could understand more fully the issues at play. Repeat.
- Once ready ask the LLM to prepare a position paper/blog.
Workflow: Given that the Karpathy’s blog suggests an upper limit on the number of sources that can be used – roughly 100-150. I have not tested this – so caveat lector. It seems prudent to have a general sense of the research space and some preliminary questions.
Improvements: I’m taking more of a research approach, so I gave the Agent the persona of an academic research librarian, had the Agent prepare a bibliography in a standardized format, and referenced in each response the original source. I also had the LLM keep copies of each question-answer in Obsidian.
New Problems: Ingesting articles requires a lot of tokens, tokens equal money. After three to four articles in 24 hours, I exhausted my allotment. Or maybe I exceeded my context window. The bottom line is that I need to figure out to optimize my use tokens to optimize cost efficiency.
One pertinent question is practically speaking how many tokens are required to set up, and then maintain, a knowledge wiki. If the cost is really quite high, then I may need to revisit the question of local versus cloud AI. Also, at least initially, Claude code is not consistently loading PDF tools requiring reloading them each session.
Be well!