About the Website

I have decided to host this website at home, not because it is easy, but because it is hard.

The DIY Method

I've decided the host this website at home. It would have been much easier to use a paid hosting service using easily updatable templates that just anyone could setup. If my goal were to stand-up a website quickly and cheaply for a customer, that could be the preferred solution from a cost point-of-view.

Goals

But for me it is all about control and the DIY spirit! Or, maybe I'm just too cheap to pay the $5 per month for a subscription service and deny myself the many pleasurable hours getting things debugged and setup myself. The truth is, there is much personal growth, satisfaction, and motivation that comes with DIY. Eventually I look to have various back-ends setup to provide some dynamic content.

RAG

One of my goals is to host a local AI chatbot RAG modelThe AI RAG (Retrieval-Augmented Generation) model improves responses by first searching a set of documents for relevant information, then using that information to generate accurate answers. It combines document retrieval with text generation to provide well-informed responses. using AnythingLLM for an embedded front-end, and a back-end of Ollama (maybe hosting Llama 3, Phi-3, or Mistral 7b), and a Chroma vector database all in a Dockerized environment accessing an NVIDIA RTX 4090 GPU for hardware accelerated inference.

The goal of the RAG based chatbot is to answer questions about this web page and any other documents I may load into the RAG model. I have the entire setup mentioned above working on an internal lab environment already. The challenge so far has been trying to get the LLMs to identify the correct source references, answer accurately, and not hallucinate. At the moment I am planning to try different chunking sizes and see if some additional meta-data sprinkled throughout the reference material might help.

One of the potential improvements over the RAG solution is to implement a knowledge graph RAG. But this is an endeavour for another day.

The Base Tech

Other than setting up the web page, I have been focusing on the technology base for home hosting. This has included setting up a Proxmox hypervisor, implementing network segmentation, securing various network segments with pfSense, and host based firewalls. Also part of the journey was setting up various Proxmox VMs, LXC containers, Docker containers, network services (i.e. DNS bind9, DHCP KEA /w DDNS, etc.), NGINX web server, CertBot/LetsEncrypt for TLS certificates, and more.

In case you're interested, this website is build around the Massively template from HTML5 UP. Since the HTML5 UP templates don't provide any automation, the templates were modified and python/shell scripts using Jinja2 were developed to assist with managing and publishing content.

On the side

Please note, this is not the focus of what I do for a living (though I've done software, integration, networking, and architecture design past). I organize teams of technical masters to solve business problems using SOTAState Of The ART. Kind of like GOAT ;) technology. This is just a side project (until the next side project comes along - and it will).

Low Power, Cost effective hosting in a Box!