StoryWerk – When Stories Become Images

A small experiment with local AI, storytelling, and the question of how text and images can be brought together.

Lately, I’ve been spending a bit more time exploring artificial intelligence again. What interests me most is what locally running AI models are capable of these days. That curiosity led to my latest project: StoryWerk.

The basic idea is quite simple: An application where I can write stories, edit text with the help of AI, and generate images to go along with individual scenes. What I find particularly interesting is the possibility of using the actual content of a story as the starting point for image generation.

StoryWerk is still a long way from being a finished application, though. Right now, it’s a feasibility study and an experimental prototype. I want to try out different technical approaches, gain some experience, and see what actually works in practice and what perhaps doesn’t.

From a Written Scene to an Image

One of the questions I’m particularly interested in with StoryWerk is how well AI can actually understand what’s happening in a story.

Can it figure out from a paragraph which characters are involved, where a scene takes place, and what kind of mood the text is supposed to convey? And can it turn all of that into an image description that an image generator can actually work with?

That’s exactly what I want to find out.

The idea is that I select a passage of text and a language model creates a suitable description for image generation. This description is then passed on to an image model. Ideally, the result is an illustration that actually matches the scene I’ve written.

Of course, it’s not always that simple. Something that seems perfectly clear in a written passage might be interpreted quite differently by an AI. Then there are questions about keeping characters recognizable, getting certain details right, maintaining a consistent visual style across multiple scenes, and so on.

Those are exactly the kinds of challenges I find interesting. After all, the point of this project isn’t to generate random pictures as quickly as possible. It’s about understanding how well these different technologies can work together.

Keeping Everything as Local as Possible

Another thing that’s important to me with StoryWerk is running the AI locally.

I want to be as independent of cloud services and external APIs as possible. That’s why I’m using language models that run directly on my own computer, along with local image generation based on Stable Diffusion.

I’m developing StoryWerk as a desktop application. The idea is to bring the different features together in a single interface, so I don’t have to constantly switch between separate programs.

The prototype already includes a text editor, some initial AI-assisted writing and editing features, and a section for generating images.

All of this is still under development. Some things already work pretty well, while others still need quite a bit of work. But that’s what a prototype is for, after all.

AI as a Tool, Not a Replacement

There’s also something I want to make clear about what StoryWerk is not supposed to be.

I don’t want it to replace writing stories yourself, drawing, painting, or creating artwork by hand. And I certainly don’t want to hand over the entire creative process to an AI.

People who enjoy writing or drawing themselves should absolutely continue doing so. AI can make suggestions, rephrase text, or generate images. But it doesn’t have the personal ideas, experiences, and intentions that a person brings to their own creative work.

For me, AI is therefore primarily a tool. Maybe it can help develop an idea further, suggest a different way of phrasing something, or make it easier to visualize a scene. But whether I accept a suggestion, change it, or reject it completely is still entirely up to me.

The same goes for AI-generated images. They can be an interesting addition, but in my opinion, they’re not better than something someone has drawn or created themselves.

The actual creative work should remain in human hands. For me, StoryWerk is a way to explore how AI can support that work.

Let’s See Where It Goes

I can’t really say yet how far I’ll take StoryWerk.

There are quite a few ideas I’d like to try out, and plenty of unanswered questions as well. Longer stories, recurring characters, and getting the atmosphere of a particular scene just right will probably come with their own challenges.

I’m deliberately leaving open the question of whether this will eventually become an application I make available to others. That’s also why I haven’t planned a release date or a fixed set of features.

For now, I’m simply enjoying experimenting with the different possibilities, trying out new approaches, and learning a few things along the way.

And even if StoryWerk ends up remaining just a prototype, the time spent on it won’t have been wasted. Sometimes, with projects like this, the most interesting part isn’t the finished software, but everything you discover while working on it.