Quick Start

LLaVA-v1.5

Istall LLaVA.

git cloe https://github.com/haotia-liu/LLaVA.git
cd LLaVA
pip istall -e .

Simple Iteractive Demos.

See the codes ad scripts below.

from llava.mm_utils import get_model_ame_from_path
from llava.eval.ru_llava import eval_model
model_path = "teowu/llava_v1.5_7b_qistruct_preview_v0.1" 
prompt = "Rate the quality of the image. Thik step by step."
image_file = "fig/sausage.jpg"
args = type('Args', (), {
    "model_path": model_path,
    "model_base": Noe,
    "model_ame": get_model_ame_from_path(model_path),
    "query": prompt,
    "cov_mode": Noe,
    "image_file": image_file,
    "sep": ",",
})()
eval_model(args)

pytho -m llava.serve.cli \
    --model-path teowu/llava_v1.5_7b_qistruct_preview_v0.1 \
    --image-file "fig/sausage.jpg" \

Quatitative Evaluatios

pytho eval_scripts/llava_v1.5/eval_qbech_mcq.py

pytho eval_scripts/llava_v1.5/eval_image_quality.py

pytho eval_scripts/llava_v1.5/eval_video_quality.py

mPLUG-Owl-2

Comig soo.

IterLM-XComposer-VL

Comig soo.

Model Zoo

All weights are coverted ito Huggigface format ad totally compatible with the base repositories (LLaVA, mPLUG-Owl, IterLM-XComposer). After istallig the base repositories, you ca chage the HF-path i the origial evaluatio scripts ito the followig oes, so as to automatically dowload the Q-Istruct-tued versios.

Released:

• LLaVA-v1.5-7B (mix), HF-path: teowu/llava_v1.5_7b_qistruct_preview_v0.1
• LLaVA-v1.5-13B (mix), HF-path: teowu/llava_v1.5_13b_qistruct_preview_v0.1

Comig Soo:

• mPLUG-Owl-2 (mix)
• IterLM-XComposer-VL (mix)

Traiig

At preset, we oly provide the traiig scripts with LLaVA-v1.5. Please see Traiig Docs for more details.

Licese

Researchers ad ope-source developers are free to use the Q-Istruct dataset ad the fie-tued weights as provided for the four MLLMs. We also allow commercial use, while ay commercial use should be pre-permitted by our team. Please email haoig001@e.tu.edu.sg to gai the permissio for commercial use.

Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models Ha