- What is foundation model
- What is diffusion model
- Become familiar with Runway and understand how to run new models/tools in the browser.
- Explore DALL-E 2 image generation tool
- Machine Learning En Plein Air: Building accessible tools for artists by Cristóbal Valenzuela
- Runway: Adding artificial intelligence capabilities to design and creative platforms by Cristóbal Valenzuela, Alejandro Matamala, and Anastasis Germanidis
- Foundation models on Wikipedia
- On The Opportunities and Risks of Foundation Models
- What is Diffusion model?
- High-Resolution Image Synthesis with Latent Diffusion Models
- The research origins of Stable Diffusion
- RunwayML.com
- AI Magic Tools
- Introductory Guide to ML Lab
- Introductory Guide to Video Editing
- Model Directory
- RunwayML YouTube Channel
- Lexica: The Stable Diffusion search engine
- Image to Music demo
- runwayml/stable-diffusion-v1-5 on Hugging face
- Midjourney
- Where to find stock videos/image: Pexel
- The new era of foundation models, summarized
- The research origins of Stable Diffusion
- Machine Learning En Plein Air: Building accessible tools for artists by Cristóbal Valenzuela
-
Pick one of the following tools to generate any media(image, video, text, 3D texuture). You can choose a workflow that you worked with in class or a different one. Feel free to try more than one if you like.
- Runway: AI Magic Tools, Models Playground, Video Editor
- DALL-E: https://labs.openai.com
Consider the following questions:
- Describe the results of working with the tool, do they match your expectations?
- Can you "break" the tool? In other words, use it in a way that it was intended for and what kinds of results do you get?
- Can you find any pro tips in terms of prompt engineering? Can you change your prompt to make the generated results better?
- You can also pick one image from lexica as a start point.
-
Document your thoughts on the above questions and your experience working with Runway/ DALL-E in a blog post. Include screenshots and screen captures of your workflow. Compare and constrast working with Runway as a tool for machine learning as related to ml5.js, python, and any other tools explored this semester. Link from the homework wiki.