THU | Images & Video

September 26, 2024

9:15–12:00

Room 2420 (Marsio)

Inspiration #

External files #

Take a look at this arrow icon on the p5js web editor. Click it!

This reveals something very importatnt about working with p5.js. Our code is part of a website with some additional files.

index.html The html file that provides the structure for the website. It uses the HTML markup language to describe how the page is structures.
sketch.js This is the file we have been working with so far. The actual JavaScript file that has all of our code.
style.css This CSS file is used to describe how all of the elements on the webpage should look like

We can add additional files here (images, videos, sound, text etc.) if we want to use them in our code.

Click the arrow icon pointing down next to where it says “Sketch Files”
“Create Folder”
Name the folder data. Note that you can name this however you want but I’m using data as that name is used in Processing and other creative coding environments.
Select the folder you just created and press the arrow icon next to it
This menu shows options to create or upload files to the folder

Upload all of your files to this folder in today’s examples.

Working with images #

You can use your own images, or if you want to follow along with my examples, you can download the image below:

I also made this png image with transparent background you can use (right click and save image as):

let img;

// Loading the image file is usually done with the preload() function.
// This function makes sure that the file is loaded before it goes to setup() and draw()
function preload() {
  img = loadImage("shroom.png");
}

function setup() {
  createCanvas(512, 512);
}

function draw() {
  background(0, 20);
  tint(255);
  image(img, 0, 0, width, height);
  tint(255, 100);
  image(img, mouseX, mouseY, 256, 256);
}

let brush;
let img;

function preload() {
  img = loadImage("shroom.png");
  brush = loadImage("brush.png");
}

function setup() {
  createCanvas(512, 512);
  imageMode(CENTER);
}

function draw() {
  background(0, 20);
  tint(255);
  imageMode(CORNER);
  tint(255);
  image(img, 0, 0, width, height);
  imageMode(CENTER);
  tint(180, 0, 0);
  image(brush, mouseX, mouseY, 32, 32);
}

get() #

The get() function in p5.js is used to get the color of an idividual pixel or a region.

This could be used for many purposes. For example, you could check the color of the pixel where the mouse cursor is.

let img;
let c; // variable to store the color

function preload() {
  img = loadImage("shroom.png");
}

function setup() {
  createCanvas(512, 512);
  strokeWeight(3);
}

function draw() {
  background(0);
  image(img, 0, 0, width, height);
  c = get(mouseX, mouseY);
  fill(c);
  circle(mouseX, mouseY, 25);
}

Or you can do the same thing we did with the random walkers last week, but pick a color for each particle based on the image.

Working with video #

Video files #

Working with video files is quite similar to working with images. The loading of the video file and enabling playback of it just needs to be done in a specific way. Once that is done, you draw the video the same way you would draw any image.

createVideo

Ypu can dowload saome videos here if you don’t have anything on your hard drive (right click –> save as):

let vid;

function preload() {
  vid = createVideo("shroom.mp4", vidLoad);
}

function setup() {
  createCanvas(512, 512);
  background(100);
}

function draw() {
  image(vid, 0, 0, width, height);
}

// This function is called when the video loads
function vidLoad() {
  vid.loop();
  vid.hide();
}

Live video #

Using video camera instead of a video file works in a similar way. You need to enable the capturing of the video in a specific way, after that it’s just a moving image.

createCapture

The p5js widget that I have been using does not work with the live video. So I will just provide the code examples and links to the projects on the p5js editor. Links will be updated after the class.

let capture;

function setup() {
  createCanvas(400, 400);
  capture = createCapture(VIDEO);
  // you can use this to hide the video preview under the canvas
  capture.hide();
}

function draw() {
  background(220);
  image(capture, 0, 0, width, (width * capture.height) / capture.width);
}

Camera Aspect Fix #

Unfortutely, the video capture is not fetched with the correct aspect ratio. This could change depending on the browser you are using too. So we need to do some extra step to make sure the video is displayed correctly. The following code access the stream object from the browser and sets the resolution to 960x540. This is done before the createCapture() function is called. It works in all browsers because it uses the native browser API.

You could use the native resolution of your camera example 1280x720, or use a ratio like 16:9. The example below is using 960x540 which is still 16:9 but a 1920x1080 divided by two. Or you can fetch the full resolution of the camera and resize it inside the setup() function.

/*

MORE INFO

https://w3c.github.io/mediacapture-main/getusermedia.html#dom-constraindouble
https://www.folkstalk.com/tech/how-to-force-a-16-9-ratio-with-getusermedia-on-all-devices-solution/
https://calculateaspectratio.com/16-9-calculator

*/

let video;

const cameraWidth = 960; // modify your resolution x here
const cameraHeight = 540; // modify your resolution y here

function setup() {
  // canvas and other p5 functions HERE

  const constraints = {
    video: { width: cameraWidth, height: cameraHeight, facingMode: "user" },
  };
  // Dont remove or change anything from the function below
  navigator.mediaDevices
    .getUserMedia(constraints)
    .then((stream) => (video.srcObject = stream))
    .then(() => new Promise((resolve) => (video.onloadedmetadata = resolve)))
    .then(() => log(video.videoWidth + "x" + video.videoHeight))
    .catch((e) => {});

  // Shows video
  video = createCapture(constraints);
  // video.hide()

  // canvas and other p5 functions HERE
}