Error logging when localizing large models into RagFlow

Written by

Audrey Miles

Updated on:July-01st-2025

A few days ago, I built a RagFlow and a QwQ-32B model based on the vLLM framework. Today, I tried to connect the QwQ-32B to RagFlow to test the function of RagFlow and the effect of building a Rag model based on it. As a result, there were a lot of errors, and I spent a long time to correct them. I will record them here.

Error: Unable to connect

First, make sure the host machine can connect to the large model service.
Log in to the ragflo w container to test the connection. Sure enough, the connection failed

docker exec -it 0b527d272baa /bin/bash curl -I http://10.10.10.10:8080

I suspected that there was something wrong with the container's network configuration, so I checked the relevant configuration file docker-compose-CN-oc9.yml, and found the problem. In order to avoid port conflicts when deploying ragflow, I changed port 80 in the original file (docker-compose.yml) to 8090, that is, port 80 of the container was mapped to port 8090 of the host, but I did not change the configuration in docker-compose-CN-oc9.yml!!!

After the modification is completed, restart the Docker service

sudo systemctl restart docker

Re-enter the ragflow container to verify connectivity

curl -X POST http://10.128.32.23:8080/v1/chat/completions \> -H "Content-Type: application/json" \> -d '{> "model": "QWQ-32B",> "messages": [{"role": "user", "content": "Hello"}]> }'

The connection was successful, and I thought everything was fine, but...

I still can't connect. I'm really frustrated.

I continued to investigate and found that although I did not set the API-Key when deploying QwQ-32B, this API-Key must be filled in when connecting with code or framework! ! ! After adding it, the connection was successful!