Building on the previous episode, we look at refactoring our background job into a more maintainable object and provide context to the LLM so that we can chain together responses for a more conversational experience.

In this episode, we look at running a self hosted Large Language Model (LLM) and consuming it with a Rails application. We will use a background to make API requests to the LLM and then stream the responses in real-time to the browser.

We can create a small python service that uses a Large Language Model (LLM) to detect if a message is spam or not. Using this service, we can tie it into our Rails application so that any comment created will be evaluated for being spam or not. We explore a few different routes on handling any messages flagged as spam.