Intro-ML-Arts-IMA-F24

Introduction

• Introduction Slides
• Example Sketches

Objectives

• Define Artificial Intelligence (AI), Machine Learning (ML), and Deep Learning (DL).
• Learn about the history of machine learning.
• Overview of applications of machine learning, especially for art and design.
• Learn about the tools for machine learning.
• Understand the difference between classification and regression.
• Understand the concept of a “machine learning model”.
• What is a “pre-trained model”?
• What does it mean to discuss the “architecture” of a machine learning model?
• Define and diagram an artificial neural network.
• Understand what ml5.js is and how it fits into the TensorFlow and open source machine learning library ecosystem.
• Learn how to create an image classifier with ml5.js and MobileNet.
• Understand how the MobileNet model was trained, specifically the origins and collection methodology for the training.

Tools

• p5.js Website
  • p5.js Reference
  • p5.js Web Editor
• ml5.js Website
  • ml5.js Reference
• TensorFlow.js Website
• ml5.imageClassifier Reference
  • ml5.imageClassifier Source Code
  • TensorFlow.js MobileNet Source Code
• TensorFlow.js models
  • TensorFlow.js demos

Code Examples

• Image Classification - Single Image
• Image Classification - Drag and Drop Image
• Image Classification - Video
• Image Classification - Video with Speech Output

p5.js review

• Learning While Making p5.js with Lauren McCarthy.
• p5.js Web Editor Overview with Cassie Tarakajian.
• Code! Programming with p5.js by Daniel Shiffman.
• Creative Coding for Absolute Beginners by Jack B. Du.

ml5.js Video Tutorials (updated ones coming soon hopefully!)

• A Beginner’s Guide to Machine Learning with ml5.js - video tutorial (This video does not reference ml5.js 1.0, refer to the ml5.js Resources Wiki page for more information.)
• ml5.js: Image Classification

Supplemental Materials

• What is machine learning? by Karen Hao.
• Algorithmic Accountability: A Primer from Data & Society.
• Seeing Machines Think by Martin Wattenberg and Fernanda Viégas.
• Weird Intelligence by Kyle McDonald.
• A Brief History of Neural Nets and Deep Learning by Andrey Kurenkov.
• ImageNet: The Data That Transformed AI Research—and Possibly the World by Dave Gershgorn.
• How we teach computers to understand pictures by Fei-Fei Li

Assignment 1

1. Create a blog (or a category on a blog) for the course. (You may use any means for publishing your assignments including, but not limited to, a GitHub markdown file, Notion page, medium post, etc.) This wiki page has resources and information on creating your own blog. Additionally, there is some information on privacy options and more at NYU’s Wordpress Knowledge Base.
2. Read A People’s Guide to AI by Mimi Onuoha and Mother Cyborg (Diana Nucera).
3. Explore ImageNet, ImageNet sample images, Kaggle ImageNet Mini 1000. What surprises you about this data set? What questions do you have?
4. Try playing Emoji Scavenger Hunt.
5. Using the code examples above, try running image classification on a variety of images. What does the model recognize properly? What does it not recognize? What other aspects of the image affect the classification, including but not limited to position, scale, lighting, etc. You are welcome to use the provided examples as they are or modify the code to create interactive experience that reflects your creativity and curiosity!
6. Document your thoughts on MobileNet and image classification in a blog post and add a link to the Assignment 1 Wiki page.