Forum | Drifting Ruby

I enjoyed episodes #445 (Streaming LLM Responses) and #521 (Model Context Protocol), and now I'm trying to combine the two so my Rails app can get responses from Ollama about data in the app itself. However, for a couple of days, I've been stuck trying to connect Ollama to the FastMcp server.
The server is available at `http://localhost:3000/mcp/sse`, but how can I tell Ollama that?

From my understanding, that is possible using a model that supports tools. However, in the examples, the tools are hardcoded in the call itself.
My goal is to do what David did in #521 when he added a custom connector to Claude, but with my Ollama model.
I'm afraid I'm missing something here, but I can't see what.

Has anyone been able to let a model hosted in Ollama consume from an MCP server?

Do I have to add the tools to my Ollama call?

def response_to(message)
uri = URI("http://localhost:11434/api/generate")
request = Net::HTTP::Post.new(uri, "Content-Type" => "application/json")
request.body = {
model: "qwen3:1.7b",
prompt: "[INST]#{message}[/INST]",
temperature: 1,
stream: true
}.to_json

Net::HTTP.start(uri.hostname, uri.port) do |http|
http.request(request) do |response|
response.read_body do |chunk|
...
end
end
end
end

Thanks for any help,
Sig

David Kimura PRO said 6 months ago :

I haven't tried using ollama yet to consume a MCP. I don't think that they have native support for it, but you may be able to get around that with an additional service that sits in front of ollama so it can call the MCP server. Alternatively, you may be able to hack something together with an ollama TOOL. However, I'd imagine that this could be a bit fragile as it goes a little beyond what the tool was really meant to do. If I understand the MCPHost correctly, it would be the endpoint that you call instead of the ollama server and it should have the ability to make calls to the MCP server and also calls to ollama. It should then give a response back to the service calling it.

0 0 0

Sigfrid PRO said 6 months ago :

Hi David,
Thanks for the reply.
If I understand well, you propose to

Install MCPHost (https://github.com/mark3labs/mcphost)
Define the following mcphost.json (in development)

{
  "mcpServers": {
    "development_server": {
      "url": "http://localhost:3000/mcp/sse"
    }
  }
}

Start MCPHost

mcphost -m ollama:qwen2.5:3b --config /path/to/mcphost.json

But, at this point, how can I call the LLM?
MCPHost is a CLI host application which, if I'm not wrong, expects to interact with ollama directly in the console.

Thanks again for the help,
Sig

0 0 0

Back to forums

Learning Paths

Video Logs new

Blog

Merchandise

Forums new

How to connect fast-mcp with ollama