-
Notifications
You must be signed in to change notification settings - Fork 0
Homework Ellen
Section 03 Info | |
---|---|
Meeting Time | Tues 9:30am - 12:00pm EST |
Location | Room 408 |
Contact |
[email protected] On weekdays I aim to respond within 24 hours |
Student Hours | Mon 1:00pm – 3:00pm Zoom Tue 1:00pm – 3:00pm Zoom or In person (Coding Lab) Sign up here |
Support |
Resident office hours (schedule) The Coding Lab (schedule or drop-in help) How to ask code-related questions: examples |
Additional Materials |
Submit creative assignments Assignment responses Our Miro board Course syllabus p5.js Reference / Web Editor The Coding Train's Intro to p5.js Allison Parrish’s Creative Coding Notes xin xin's Intro to p5.js Qtv's Creative Coding tutorials in Chinese (also on bilibili) HappyCoding.io |
- About Our Course
- Week 8 • Oct 31 • Images, Pixels, Color
- Week 9 • Nov 7 • Video and Body Point Estimation
- Week 10 • Nov 14 • Sound
- Week 11 • Nov 21 • Some Text, Mostly Data
- Week 12 • Nov 28 • Final Project Proposals
- Week 13 • Dec 5 • Final Project User Testing
- Week 14 • Dec 12 • Final Project Presentations
Upon completion of the second half of the ICM course, Media, we will be able to:
- apply fundamental programming concepts to generate and manipulate media including images, video, sound, text, and data.
- experiment with media-specific, computational techniques and apply them to our creative work.
ICM Media is a seven-week course that meets once a week in person at the scheduled time. We will have assignments to do before each class, and these assignments will build towards a final exercise to share at the end of the seven weeks.
This course embraces an active-learning approach with the belief that programming is best understood by doing. In keeping with this approach, class time will be structured as a workshop and seminar instead of a lecture, relying heavily on group discussion and participation. Weekly tutorials before class sessions will introduce computational media topics. During class, supplemental code examples will be presented and discussed before hands-on work together. Time will also be allotted to share our weekly assignments. Examples and assignments will iteratively build upon lessons learned in preceding classes.
I encourage you to reach out to me outside of class to ask questions, share ideas/feedback, and discuss topics in detail.
All assignments will be posted on this page, and all are required. Unless otherwise stated, assignments are due the night before class so that I have a chance to review your questions before we meet. All assignments are expected to be submitted by the stated deadline. If you anticipate any challenges meeting the deadline, please reach out to me so that we can consider your options together.
Before class, read and/or watch the assigned tutorials about computational media topics and programming techniques to practice together when we meet in class.
There are worksheets* (linked below) to practice techniques introduced in this course. You are expected to add links to your p5 sketches directly in these documents.
There are also open-ended exercises to explore the topics for creative expression. You are expected to document your process, include a link to your p5 sketch, and answer these questions:
- What did you do?
- What worked?
- What didn’t work, and what steps did you take to try to solve the issue?
- Did any code-related questions come up for you?
- Be sure to cite (link to) any sources that you used.
Submit these assignments using our homework form*.
Here is the spreadsheet with everyone's responses*.
A big part of learning at ITP is learning from each other. Share your work and in return you'll see everyone else's!
*Log in to your NYU account to access the document.
**If you are struggling with the weekly assignments, put your energy into writing about what didn't work and how you tried to solve it. It is totally okay to vent your frustrations. Document your work to reference later and to help you synthesize what you did and tried to do.
- More presentations
- Overview of ICM Media
- Image files
- Objects with images
- Pixel regions
- Pixel manipulation
- Syllabus resources
- Media resources
- Supported media type and formats
- Pxhere | Unsplash | Wikimedia Commons or your own?
- p5 examples
- Rotating shapes demo series
- All the ways to create images
- Getting started with image files
- Animating images
- Objects with the same image
- Working with many images
- Pixel and pixel region info
- Objects of pixel regions
- Demonstrating alpha
- Canvas pixel array
- Image pixel array - change pixel color
- Pixelated image using pixel array - just get pixel color
- Color (if we have time)
- Related projects
- gämma by p1xelfool
- Kim Asendorf
- Joiri Minaya’s #dominicanwomangooglesearch
- Photomontage by Sohei Nishino*
- Four Artworks by Kensuke Koike, Photographs by Tommy Kha for The Myth of Asian American Identity*
- The Constructed Self by Karen Navarro
- The Moment We Meet by Hsin-Chien Huang
- Joiners by David Hockney
- Garbage Selfie by Jenny Odell
- Pushin' Pusheen Pixels by August Luhrs
- 2D Water Ripple Coding Challenge (p5 version by Lisa Jamhoury)
- First 1/3 of Coding Train Coding Challenge: Slitscan Time Displacement
Effect using the
copy()
function with images - *Free NYTimes subscription via NYU
- PRACTICE • Week 8 Worksheet
- CREATE • PARTS OF A WHOLE
- Create and/or manipulate an image (or images) using pixel regions and/or
individual pixels to create an alternative reality than depicted in the
source image(s). Describe in 1-3 keywords how the result feels different
from the source image(s). Write a blog
post
documenting your process and result. Work with any kind of imagery you want.
Whatever you choose, you might consider these elements of a
photograph
in your computational approach:
- Techniques and attributes: Light, focus, time, motion, vantage point/point of view, framing, cropping, composition
- Composition: Shape, line, angle, color and tone, pattern, depth, overall composition
- Content: Subject, background, foreground, people, story, mood/feeling, symbol/metaphor, style, genre, artist's intention/purpose, meaning
- Submit on our homework form.
- Create and/or manipulate an image (or images) using pixel regions and/or
individual pixels to create an alternative reality than depicted in the
source image(s). Describe in 1-3 keywords how the result feels different
from the source image(s). Write a blog
post
documenting your process and result. Work with any kind of imagery you want.
Whatever you choose, you might consider these elements of a
photograph
in your computational approach:
- READ
- Working with video tutorial by Allison Parrish
- WATCH
- Coding Train Video 11.1 on capturing your webcam ~10min
- Video with machine learning pose estimation using ml5.js. Please note that a
new version of the ml5 project is currently underway, but this documentation
for the earlier version is still a useful introduction to the concepts:
- Coding Train videos Beginner's Guide to Machine Learning with ml5.js and Pose Estimation with PoseNet ~40min
- Coding Train video Teachable Machine: Images and Snake ~30min
- A visual and intuitive understanding of deep learning from Octavio Good (0:00 - 9:40) ~10min
- (OPTIONAL) Coding Train: Live Video and createCapture on video photo booths, pixel mirrors, painting with pixels, and slit-scan video ~1hr 20min
- Share our homework
- Video files
- Working locally with VS Code
- Webcam
- Body point estimation with the next-generation ml5.js library using Bodypose, Facemesh, and Handpose
- Related projects
- Syllabus resources
- Media resources
- Supported media type and formats
- Videvo | Videezy or your own
- Option to downsize video files: VLC (free) > File > Convert / Stream > Customize > Video codec
- p5.MediaElement methods (.play(), .stop(), .onended(), and more!)
- p5 examples
- Timing events
- Video
- Webcam
- ml5.js (ml5) library next-generation
- Bodypose Documentation
- Handpose Documentation
- Handpose parts
- Handpose keypoints
- Handpose start-stop detecting (this method is also available for Bodypose and Facemesh)
- Handpose single image
- Facemesh Documentation
- Work locally with Visual Studio Code (VS Code)
- All uploaded media files to the p5 web editor, must be < 5MB
- For working with media files that are > 5MB, use a local text editor. There are many options! VS Code provides several p5 add-ons to get started quickly.
- PART 1: Set up VS Code
- Download and install VS Code (not the full Visual Studio suite)
- Once installed, click on the Extensions icon from the left sidebar menu and search for: p5.vscode by Sam Lavigne
- PART 2: Create a p5 project
- Open the Command Palette (with command-shift-p on Mac, or ctrl-shift-p on Windows) and then start typing and select "Create p5.js Project".
- Select “New Folder” to create a folder in which to put your project’s files, and then click, “Open”
- Drag your media files into your new project folder.
- Start coding on the sketch.js file. Make sure to save as you go!
- Click "Go Live" in the bottom status bar to open your sketch in a browser. If you don’t see “Go Live”, click on the port number (something like Port : 5500) to dispose of the local server on your computer, and then start a new one by clicking “Go Live”. A browser window should open with your project.
- Return to VS Code to change code. Save the file before reloading the browser window to see the updates.
- Open your browser’s Console to see any error messages:
- Chrome: View > Developer > JavaScript Console
- Firefox: Tools > Browser Tools > Web Developer Tools > “Console” tab
- Related projects
- MAXforum: Art Of Movement Mapping: Artist/Programmer Lisa Jamhoury, Artist/Technologist Mimi Yin, and Neurobiologist Dr. Ryan York discuss the creative constraints and possibilities of working with AI and movement mapping on humans (and other organisms!)
- Doppelgänger by NUUM Collective
- Online Live Performance (OLP) by Yuguang Zhang and Nuntinee Tansrisakul
- Coding Train Videos on video photo booths, pixel mirrors, painting with pixels, and slit-scan video
- p5VideoKit - a dashboard for mixing video in the browser by John Henry Thompson
- Flickering Existence by Liu Chang
- Interactive Art Works by Danny Rozin
- Más Que la Cara by Zach Lieberman
- Aspire Mirror by Joy Buolamwini
- Maya Man PoseNet Sketchbook
- Text Rain by Romy Achituv & Camille Utterback
- Body, Movement, Language: AI Sketches With Bill T. Jones
- Pose estimation examples listed on TensorFlow.org (scroll down)
- Charlie Gerard Quaruntine and Beat pose
- Vibert Thio ITP Web Olympic
- Richard Yee Touch Type: Interactive Typography
- Casher's Drawing with Body
- Finger Talk by Chelsi Cocking, Lingdong Huang, and Zach Lieberman
- AI ‘Emotion Recognition’ can’t be Trusted by James Vincent
- No worksheet this week!
- CREATE • BODY
- The idea this week is to center an experience around the body--yours,
someone else's, or a specific audience. Choose the tools and techniques that
make the most sense for the concept, keeping in mind that body point
estimation extends choices for interactive design beyond the mouse and
keyboard, which prioritize the hands and fingers for communication and
expression in a very specific way. Some questions to start brainstorming:
- What are the private, public, and performative aspects of bodies?
- How are bodies imagined, idealized, reflected, altered?
- What is the personal and cultural significance of masks for ritual, to explore identity, hide, entertain, frighten...
- In what ways can you use the body for expressive input? Consider choices for the part(s), position, distance between parts, direction of movement, and speed.
- Submit on our homework form.
- The idea this week is to center an experience around the body--yours,
someone else's, or a specific audience. Choose the tools and techniques that
make the most sense for the concept, keeping in mind that body point
estimation extends choices for interactive design beyond the mouse and
keyboard, which prioritize the hands and fingers for communication and
expression in a very specific way. Some questions to start brainstorming:
- WATCH • Coding Train Videos 17.1 - 17.3, 17.5 - 17.8 on sound (code) ~90min
- Share our homework
- Quick survey for next week
- Mic input
- Sound output (samples and oscillators)
- Making notes
- Timing and patterns
- Related projects
- Syllabus Resources
- Media resources
- Record or generate your own!
- Orchestral Sample Library
- Another Orchestral Sample Library
- Free Music Archive
- Freesound
- BBC Sound Effects
- Sound of Picture
- Blackhole for routing computer sound
- Supported media type and formats
- Audacity for audio editing
- media.io for converting media files
- p5 examples
- Mic input
- Sound output
- Making notes
- Pitch slider (out of tune)
- Note ratios for C major
- Multiple musical scales
- Melody with steps and leaps
- Random diatonic chords
- Timing and patterns
- Related projects
- NYTimes Magazine Sonic Voyages (New York: Laughing rats!) browser version*
- Can Brown Noise Turn Off Your Brain?*
- The Handbook for Sonic Happiness
- jazz.computer
- Plink by DinahMoe
- Patatap and Typatone
- Writing Music in Light
- Music Eclipticalis by Brian Foo
- Troupe-like in p5.js by Cassie Tarakajian
- ...based on Trope app by Brian Eno and Peter Chilvers
- Symphonies of the Planets by Joohyun Park
- Chrome Music Lab
- PoseNet-related Sidewalk Orchestra by Cristobal Valenzuela, Code Music by Tero Parviainen, Semi-Conductor, and Body Synth
- *Free NYTimes subscription via NYU
- PRACTICE • Week 10 Worksheet
- COMPOSE • Create a 30-60s sound composition or generative music system. Option
to work with partner. No visuals. Just sound. Some ideas to get started:
- Design a melody using this process and figure out an algorithmic way to generate it. Or, How Generative Music Works by Tero Parviainen.
- Record bits of spoken word and loop them to create music. See "Record and loop" example above. Why repeating words sound like music to your brain.
- Use sound samples and manipulate their playback rate to control pitch instead of the oscillator.
- Try implementing a different scale with different pitch ratios. More about scales and Music Theory: The TL;DR Version 2022.
- Play with Timbre and make use of p5 Sound's post-processing features: Delay, Filter, Reverb, Convolver, etc.
- Maybe the composition tells a symphonic story with several characters.
- Compose a sonic piece that is not music.
- Use audio navigational cues to implement a vision accessibility-feature for a video game. Student example by Kea Lia Pedersen.
- Submit on our homework form.
- READ
- Allison Parrish's tutorial on Text and Type
- Review these syllabus examples and tutorials for Text and Data. We'll cover a couple topics, but you might want to go further with the materials posted here.
- WATCH
- Coding Train videos 17.9-17.11 on sound analysis (code) ~36min
- Listen to homework
- Data
- of Sound
- of Text
- as JSON
- from API
- Final project overview
- Sound
- p5 examples
- Related projects
- Text
- Syllabus resources
- Font resources
- p5 examples
- Array vs string
- Dancing characters
- Random story starters
- Draw with lines from a poem
- Book word and keyword count
- Draw one word at a time
- Carrie Wang's Scrolling Text and Drawing the Text Array
- Allison Parrish's Interactive Type Examples
- Generative Design P.3. Type Examples
- Related projects
- Data & APIs
- Syllabus resources
- JSON resources
- Working with JSON
- JSON Formatter Chrome Extension
- Formatting and Validating JSON
- Coding Train Intro to Data and APIs (includes working with Wordnik, NYTimes, Giphy, and Wikipedia APIs)
- p5 examples
- Data sources
- Related projects
- Shirley Wu
- Jason Snell
- The Library of Missing Datasets by Mimi Onuoha
- Listen to Wikipedia by Stephen LaPorte and Mahmoud Hashemi
- See, also: A collection of our favorite visualizations built on Wikipedia data, curated by Hatnote
- Numbers Series: performance and sound installations by Mendi + Keith Obadike, related article: Reading the Numbers of Stop-and-Frisk
- We Need Us by Julie Freeman
- The True Colors of America’s Political Spectrum Are Gray and Green*
- Dear Data by Giorgia Lupi and Stefanie Posavec
- Sleep Drawings by Laurie Frick
- Visual Essays by The Pudding
- Heartmonic (heart+harmonic) and Eunoia (Version 2) by Lisa Park
- Engaging the Data Science Community with Met Open Access API
- 45 Ways to Communicate Two Quantities
- *Free NYTimes subscription via NYU
- DESCRIPTION
- Your final project is an open-ended creative project that builds off or is
inspired by the concepts in this class. It is also an opportunity to push
your abilities to produce something that utilizes what you have learned. The
final project includes:
- presenting a proposal to our class in Week 12,
- user testing a prototype of the project with our class in Week 13,
- and documenting your project in a blog post (or Notion, Google Doc, etc) including your code contributions to the project, when the project is shared during final presentations in Week 14.
- There is no requirement to use a particular aspect of programming. Focus on an idea that excites you, and choose the best programming concepts and tools to help you realize it.
- You can take something you've already made and develop it further or create something entirely new.
- Final projects can be one part of a larger project integrated with a different class.
- Final projects can be collaborations with anyone in any class. Group projects are welcome and encouraged, and it is expected that everyone contribute their own code to the project.
- It’s okay to keep things simple and small in scope. If your project idea is a big one, consider documenting the larger idea but implementing just a small piece of it.
- Also consider making a final project for a small audience such as our class or for a single person like a family member, friend, or yourself as a way to focus your idea and design process.
- Your final project is an open-ended creative project that builds off or is
inspired by the concepts in this class. It is also an opportunity to push
your abilities to produce something that utilizes what you have learned. The
final project includes:
- EXAMPLES from ITP/IMA/IMA Low Res Spring Show 2020 | Winter Show 2020 | Spring Show 2021
- FINAL PROJECT PROPOSAL
- PART 1 • Research your project and collect some notes, such as:
- Inspirations • How did you become interested in this idea? quotes, photographs, products, projects, people, music, political events, social ills
- Source material • Drawings, images, videos, sounds, text
- Code • Your own sketches and/or other people's sketches
- Ideas for title • And maybe a 1-sentence description?
- Context • What is the story? Who's it for? How will people experience it? Is it interactive? Is it practical? Is it for fun? Is it emotional? Is it to provoke something?
- Questions for your classmates • What are you unsure of, conceptually and technically? On what aspect(s) would you like feedback?
- PART 2 • Add two (2) to three (3) slides to this deck to share your proposal. Consider using the list above as an outline to illustrate what’s in your head(s), e.g. include some keywords, images, or embed video, etc. Everyone will have ~4 minutes each to share in our next class.
- PART 1 • Research your project and collect some notes, such as:
- Final project proposals
- Discuss final project user testing
- ml5's ImageClassifier
- On saving the current canvas as an image in p5
- saveCanvas()
- For timing canvas saves see p5 examples for Week 9
- Generate a gif from a p5 sketch and save it to a file with saveGif()
- A geometry of music: harmony and counterpoint in the extended common practice (online book available in the library)
- Coding Train: Circle [shape] morphing and related examples
- On mirrors
- Danny Rozin's mirrors including Self Centered Mirror 2003
- Yang Yang's mirrors: No One Wanna See U and Simple Mirror
- NUUM collective's Doppelgänger and related conversation MAXforum: Art Of Movement Mapping
- On AI conversations
- REBUILDING BABEL by Karina Chow, Dror Margalit, Maryia Markhvida
- Carrie Wang's Lost and Found and related course on chatbots
- Stephanie Dinkins and Bina48
- Change the pitch of a sound with .rate() and see examples in Week 10 for doing so with specific ratios to make notes
- On working with ml5 pose estimation models and text
- For those interested in projection: p5.mapper
- On working with next-gen ml5 on a larger Canvas: Get webcam + next-gen ml5 f23 (use Chrome)
- FINAL PROJECT PROTOTYPE
- PART 1 • Create a working prototype of your final project to user test in
class next week.
- If your project is screen-based, be prepared to share a live sketch, full-screen if possible.
- If your project involves some type of physical interaction, ideally bring it to class or show a short video (~2 min)
- PART 2 • Consider the kind of feedback that will be most useful for your project's development. Prepare some open-ended questions for your testers that focus on specific aspects of your project that you’re unsure of or want to know more about, e.g. “What did you take away from this project?” instead of “Do you get that my project is about XYZ?”
- Submit on our homework form.
- PART 1 • Create a working prototype of your final project to user test in
class next week.
- User testing
- Discuss final project documentation and presentations
- Course evaluation
- FINAL PROJECT
- PART 1 • Complete your final project.
- PART 2 • Write your final project documentation in the form of a blog post
(or Notion, Google Doc, etc). It's up to you to figure out how to best
document your project. Here are some guidelines if you aren't sure what to
include:
- Title and a brief written description
- Visual documentation such as a sketch running online, images, video, etc.
- References such as links to related projects, code samples, etc.
- What were some of the successes and challenges? What did you learn?
- How do you feel about WHY you want to use code in your work now compared with the beginning of the semester?
- If you are struggling with your project and can't get things to work, you should feel free to put your energy into writing about what didn't work (and vent any frustrations).
- Next steps to further develop the project.
- If the project is a collaboration, each person needs to contribute code and submit a post documenting their contributions., to the project.
- Be sure to cite the sources of any material / code you use with the exception of examples provided by the instructors or demonstrated for the course.
- PART 3 • Prepare a ~4-5 minute presentation to demonstrate what your project
does that emphasizes its computational aspects. Feel free to include any
questions to direct feedback after your presentation.
- If your project is interactive, be prepared to share a live sketch online. Aim to share a project running full-screen.
- If your project involves physical interaction, as a backup, prepare a short video (~2 min) of the experience.
- Submit on our homework form.
- Welcome and general notes
- Final project presentations