Motio LoRAs

Motio LoRAs allow addig specific types of motio to your aimatios.

Curretly the followig types of motio are available for models usig the guoyww/aimatediff-motio-adapter-v1-5-2 checkpoit.

• Zoom I/Out
• Pa Left/Right
• Tilt Up/Dow
• Rollig Clockwise/Aticlockwise

Please refer to the AimateDiff documetatio for iformatio o how to use these Motio LoRAs.

Istall requiremets:

pip istall trasformers peft diffusers -U

import torch
from diffusers import MotioAdapter, AimateDiffPipelie, DDIMScheduler
from diffusers.utils import export_to_gif
from modelscope import sapshot_dowload

model_dir = sapshot_dowload("Shaghai_AI_Laboratory/aimatediff-motio-adapter-v1-5-2")
# Load the motio adapter
adapter = MotioAdapter.from_pretraied(model_dir)
# load SD 1.5 based fietued model
model_id = sapshot_dowload("wyj123456/Realistic_Visio_V5.1_oVAE")
pipe = AimateDiffPipelie.from_pretraied(model_id, motio_adapter=adapter)
lora_dir = sapshot_dowload("Shaghai_AI_Laboratory/aimatediff-motio-lora-tilt-dow")
pipe.load_lora_weights(lora_dir, adapter_ame="tilt-dow")
pipe.set_adapters(["tilt-dow"], adapter_weights=[1.0])
scheduler = DDIMScheduler.from_pretraied(
    model_id, subfolder="scheduler", clip_sample=False, timestep_spacig="lispace", steps_offset=1
)
pipe.scheduler = scheduler

# eable memory savigs
pipe.eable_vae_slicig()
pipe.eable_model_cpu_offload()

output = pipe(
    prompt=(
        "masterpiece, bestquality, highlydetailed, ultradetailed, suset, "
        "orage sky, warm lightig, fishig boats, ocea waves seagulls, "
        "ripplig water, wharf, silhouette, seree atmosphere, dusk, eveig glow, "
        "golde hour, coastal ladscape, seaside sceery"
    ),
    egative_prompt="bad quality, worse quality",
    um_frames=16,
    guidace_scale=7.5,
    um_iferece_steps=25,
    geerator=torch.Geerator("cpu").maual_seed(42),
)
frames = output.frames[0]
export_to_gif(frames, "aimatio.gif")

Motion LoRAs Motion LoRAs allow adding specific types of motion to your animations. Currently the