Hi Pierre,
This project instantly caught my attention because it reminds me of an interactive, browser-based AI art ritual I helped an artist prototype recently, where the real challenge wasn’t the models or the visuals, but making the experience feel intentional and human rather than technical, and that’s very much the direction I sense you’re aiming for here.
I’d probably approach this as one continuous experience instead of separate features, using something like Next.js with React to keep the flow smooth and lightweight, and Three.js for the speaking character so we have tight control over animation and audio response but would amplitude-based lip sync be acceptable for an MVP if it keeps things stable and performant?
For the dialogue, I’d lean toward a self-hosted open-weight LLM like Llama or Mistral served via vLLM, mainly because they behave well with structured, does that sound good to you?
And for the poetic prompting thing do you want the dialogue to stay more symbolic throughout, or gradually become clearer and more literal toward the end?
For voice, something like Coqui or Piper could work nicely since it’s fully open-source and can directly drive the character animation , a slightly imperfect but would organic voice texture okay for the ritual tone you’re imagining?
On the image side, I’ve trained LoRAs for artist-specific styles before, and I’ve found that consistency comes from careful dataset curation. so I’d suggest an SDXL base with a dedicated LoRA trained on your artwork however should the style subtly guide the image, or fully dominate it as a signature look? Do you want fixed style strength per session or adjustable ranges? Would seed reproducibility be exposed internally only, or visible in the admin view?
For storage and future integration, I’d keep everything cleanly structured and exportable, with a simple password-protected admin panel for viewing and downloads , just that like does this admin space need any curatorial context later, or strictly raw data access?
On deployment, Docker Compose on Hetzner would keep updates and model swaps predictable or are you already running GPUs there, and if so what VRAM are we working with? Would you prefer a single GPU setup or something expandable later?
Given the custom LoRA training, 3D interaction, and self-hosted AI stack, I’d estimate this in the £2.4k–£2.7k range depending on iteration depth and polish. If you’re open to it, I’d be happy to jump on a short call or put together a quick visual walkthrough of how I’d structure the system end-to-end — would that help clarify things on your side before moving forward?