MidJourney Diffusion and PublicPrompts.art

Two fun things in this post to take away.

• The midjourney-v4-diffusion model is awesome!
• If you copy/paste the prompts below in a text file you can load up your little computer with a bunch of work.

Prompthero released a model the other day based on MidJourney AI. I wanted to try it out on a bunch of different prompts to see how it compares with the default v1.5 Stable Diffusion model.  I found that I really like the results.

prompthero/midjourney-v4-diffusion:

prompthero/midjourney-v4-diffusion · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

I took all the prompts from publicprompts.art that did not need their own model, used the docs for InvokeAI to come up with a nice prompt per line and then let my little M1 MacBook run overnight.

To run a bunch of prompts in a text file add the --from_file argument on the command line:

Note: I set mdjrny-v4.ckpt to be my default model in InvokeAI vs stable diffusion v1.5

Note 2: Add --outdir and a specific directory to export the images, otherwise they end up in your InvokeAI/outputs directory by default.

python scripts/invoke.py --from_file ./prompts.txt

InvokeAI docs for running it in the command line:

CLI - Stable Diffusion Toolkit Docs

Stable Diffusion Toolkit Docsmauwii

Prompts.txt [original and modified]

side note: the extra arguments behind the prompt are case sensitive [use ddim vs. DDIM]

Original prompts from publicprompts.art:

cute kawaii Squishy elephant plush toy, realistic texture, visible stitch line, soft smooth lighting, vibrant studio lighting, modular constructivism, physically based rendering, square image -C13.0 -Addim -s35

tiny cute 3D felt fiber toy frog, made from Felt fibers, a 3D render, trending on cgsociety, rendered in maya, rendered in cinema4d, made of yarn, square image -C13.0 -Addim -s25

Multiple layers of silhouette mountains, with silhouette of big rocket in sky, sharp edges, at sunset, with heavy fog in air, vector style, horizon silhouette Landscape wallpaper by Alena Aenami, firewatch game style, vector style background -C13.0 -Addim -s35 -W704 -H384

tiny cute ninja toy, standing character, soft smooth lighting, soft pastel colors, skottie young, 3d blender render, polycount, modular constructivism, pop surrealism, physically based rendering, square image -C13.0 -Addim -s35

Retro comic style artwork, highly detailed James Bond, comic book cover, symmetrical, vibrant -C13.0 -Addim -s35

Hypnotic illustration of a Halloween pumpkin, hypnotic psychedelic art by Dan Mumford, pop surrealism, dark glow neon paint, mystical, Behance -C13.0 -Addim -s35

hologram of a wolf floating in space, a vibrant digital illustration, dribbble, quantum wavetracing, black background, behance hd -C13.0 -Addim -s35

Die-cut sticker, Cute kawaii flower character sticker, white background, illustration minimalism, vector, pastel colors -C13.0 -Addim -s50

Closeup face portrait of a black girl wearing crown of flowers, smooth soft skin, big dreamy eyes, beautiful intricate colored hair, symmetrical, anime wide eyes, soft lighting, detailed face, by makoto shinkai, stanley artgerm lau, wlop, rossdraws, concept art, digital painting, looking into camera -C13.0 -Ak_euler_a -s50

100mm photo of isometric floating island in the sky, surreal volcano, intricate, high detail, behance, microworlds smooth, macro sharp focus, centered -C13.0 -Ak_euler_a -s50

gold dia de los muertos pendant, intricate 2d vector geometric, cutout shape pendant, blueprint frame lines sharp edges, svg vector style, product studio shoot -C13.0 -Ak_euler_a -s50

2d ferocious lion head, vector illustration, angry eyes, football team emblem logo, 2d flat, centered -C13.0 -Ak_euler_a -s50

Funko pop Yoda figurine, made of plastic, product studio shot, on a white background, diffused lighting, centered -C13.0 -Ak_euler_a -s50

Tiny cute isometric temple, soft smooth lighting, soft colors, soft colors, 100mm lens, 3d blender render, trending on polycount, modular constructivism, blue blackground, physically based rendering, centered -C13.0 -Ak_euler_a -s50

kawaii low poly panda character, 3d isometric render, white background, ambient occlusion, unity engine -C13.0 -Ak_euler_a -s50

Tiny cute isometric living room in a cutaway box, soft smooth lighting, soft colors, purple and blue color scheme, soft colors, 100mm lens, 3d blender render -C10.0 -Ak_euler_a -s50

3d fluffy llama, closeup cute and adorable, cute big circular reflective eyes, long fuzzy fur, Pixar render, unreal engine cinematic smooth, intricate detail, cinematic -C13.0 -Ak_euler_a -s50

Tiny cute isometric porcelain cup of coffee, soft smooth lighting, with soft colors, 100mm lens, 3d blender render, trending on polycount, modular constructivism, blue background, physically based rendering, centered -C13.0 -Ak_euler_a -s50

Same prompts, but modified with mdjrny-v4 style added at the beginning.

mdjrny-v4 style cute kawaii Squishy elephant plush toy, realistic texture, visible stitch line, soft smooth lighting, vibrant studio lighting, modular constructivism, physically based rendering, square image -C13.0 -Addim -s35

mdjrny-v4 style tiny cute 3D felt fiber toy frog, made from Felt fibers, a 3D render, trending on cgsociety, rendered in maya, rendered in cinema4d, made of yarn, square image -C13.0 -Addim -s25

mdjrny-v4 style Multiple layers of silhouette mountains, with silhouette of big rocket in sky, sharp edges, at sunset, with heavy fog in air, vector style, horizon silhouette Landscape wallpaper by Alena Aenami, firewatch game style, vector style background -C13.0 -Addim -s35 -W704 -H384

mdjrny-v4 style tiny cute ninja toy, standing character, soft smooth lighting, soft pastel colors, skottie young, 3d blender render, polycount, modular constructivism, pop surrealism, physically based rendering, square image -C13.0 -Addim -s35

mdjrny-v4 style Retro comic style artwork, highly detailed James Bond, comic book cover, symmetrical, vibrant -C13.0 -Addim -s35

mdjrny-v4 style Hypnotic illustration of a Halloween pumpkin, hypnotic psychedelic art by Dan Mumford, pop surrealism, dark glow neon paint, mystical, Behance -C13.0 -Addim -s35

mdjrny-v4 style hologram of a wolf floating in space, a vibrant digital illustration, dribbble, quantum wavetracing, black background, behance hd -C13.0 -Addim -s35

mdjrny-v4 style Die-cut sticker, Cute kawaii flower character sticker, white background, illustration minimalism, vector, pastel colors -C13.0 -Addim -s50

mdjrny-v4 style Closeup face portrait of a black girl wearing crown of flowers, smooth soft skin, big dreamy eyes, beautiful intricate colored hair, symmetrical, anime wide eyes, soft lighting, detailed face, by makoto shinkai, stanley artgerm lau, wlop, rossdraws, concept art, digital painting, looking into camera -C13.0 -Ak_euler_a -s50

mdjrny-v4 style 100mm photo of isometric floating island in the sky, surreal volcano, intricate, high detail, behance, microworlds smooth, macro sharp focus, centered -C13.0 -Ak_euler_a -s50

mdjrny-v4 style gold dia de los muertos pendant, intricate 2d vector geometric, cutout shape pendant, blueprint frame lines sharp edges, svg vector style, product studio shoot -C13.0 -Ak_euler_a -s50

mdjrny-v4 style 2d ferocious lion head, vector illustration, angry eyes, football team emblem logo, 2d flat, centered -C13.0 -Ak_euler_a -s50

mdjrny-v4 style Funko pop Yoda figurine, made of plastic, product studio shot, on a white background, diffused lighting, centered -C13.0 -Ak_euler_a -s50

mdjrny-v4 style Tiny cute isometric temple, soft smooth lighting, soft colors, soft colors, 100mm lens, 3d blender render, trending on polycount, modular constructivism, blue blackground, physically based rendering, centered -C13.0 -Ak_euler_a -s50

mdjrny-v4 style kawaii low poly panda character, 3d isometric render, white background, ambient occlusion, unity engine -C13.0 -Ak_euler_a -s50

mdjrny-v4 style Tiny cute isometric living room in a cutaway box, soft smooth lighting, soft colors, purple and blue color scheme, soft colors, 100mm lens, 3d blender render -C10.0 -Ak_euler_a -s50

mdjrny-v4 style 3d fluffy llama, closeup cute and adorable, cute big circular reflective eyes, long fuzzy fur, Pixar render, unreal engine cinematic smooth, intricate detail, cinematic -C13.0 -Ak_euler_a -s50

mdjrny-v4 style Tiny cute isometric porcelain cup of coffee, soft smooth lighting, with soft colors, 100mm lens, 3d blender render, trending on polycount, modular constructivism, blue background, physically based rendering, centered -C13.0 -Ak_euler_a -s50

Results in order from the prompts in the text

The 1st image would be default SD, 2nd with the MidJourney style added.

Elephant:

Frog

Wallpapers

Ninja

Bond comic

Pumpkin

Wolf

Cute Sticker

Anime

Island

Skull

Lion Sports logo

Yoda

Castle

Panda

Isometric Room

Llama

Coffee

Final Thoughts

It is kind of cool that all these models are coming online, each one just a little better than the other.  

I ran other prompts through the model and am super happy with the results.  I will most likely keep this as my default for now until something new replaces it, as just the normal prompts work.

If you have any questions or comments, please feel free to reach out to me on Mastodon.

Jeremy