模型文件和权重，请浏览“模型文件”页面获取。
当前模型的贡献者未提供更加详细的模型介绍，但是您可以通过如下git clone命令，或者ModelScope SDK来下载模型。
Clone with HTTP
git clone https://www.modelscope.cn/hankwang/chatglm.cpp.git
如果您是本模型的贡献者，我们邀请您根据模型贡献文档说明，及时完善模型卡片内容。
API Server
We support various kinds of API servers to integrate with popular frontends. Extra dependencies can be installed by:
pip install 'chatglm-cpp[api]'
Remember to add the corresponding CMAKE_ARGS to enable acceleration.
LangChain API
Start the api server for LangChain:
MODEL=./chatglm-ggml.bin uvicorn chatglm_cpp.langchain_api:app --host 127.0.0.1 --port 8000
Test the api endpoint with curl:
curl http://127.0.0.1:8000 -H 'Content-Type: application/json' -d '{"prompt": "你好"}'
Run with LangChain:
>>> from langchain.llms import ChatGLM
>>>
>>> llm = ChatGLM(endpoint_url="http://127.0.0.1:8000")
>>> llm.predict("你好")
'你好?！我是人工智能助手 ChatGLM2-6B，很高兴见到你，欢迎问我任何问题。'
For more options, please refer to examples/langchain_client.py and LangChain ChatGLM Integration.
OpenAI API
Start an API server compatible with OpenAI chat completions protocol:
MODEL=./chatglm-ggml.bin uvicorn chatglm_cpp.openai_api:app --host 0.0.0.0 --port 8000
或
MODEL=./chatglm-ggml.bin daphne chatglm_cpp.openai_api:app --b 127.0.0.1 --port 8000
Test your endpoint with curl:
curl http://127.0.0.1:8000/v1/chat/completions -H 'Content-Type: application/json' \
-d '{"messages": [{"role": "user", "content": "你好"}]}'
Use the OpenAI client to chat with your model:
>>> import openai
>>>
>>> openai.api_base = "http://127.0.0.1:8000/v1"
>>> response = openai.ChatCompletion.create(model="default-model", messages=[{"role": "user", "content": "你好"}])
>>> response["choices"][0]["message"]["content"]
'你好?！我是人工智能助手 ChatGLM2-6B，很高兴见到你，欢迎问我任何问题。'
For stream response, check out the example client script:
OPENAI_API_BASE=http://127.0.0.1:8000/v1 python3 examples/openai_client.py --stream --prompt 你好
With this API server as backend, ChatGLM.cpp models can be seamlessly integrated into any frontend that uses OpenAI-style API, including mckaywrigley/chatbot-ui, fuergaosi233/wechat-chatgpt, Yidadaa/ChatGPT-Next-Web, and more.
评论