06MA4010991AP
Computer Game Design

Week 5

Processing Imaging Concept (reference - http://sweb.cityu.edu.hk/sm3123/class02/class02.html)

PImage is the object class for images in Processing. It contains a 1-dimensional array of all pixels in the image.

>>> >>>

In order to extract one pixel from the image, the general formula is : y * width + x, where y is the row number and x is the column number (both count from zero). And img.get(x,y) do the same job too!

So Basically you can scan all the pixels be two methods below:

1) Read the pixels by array index



 
for(int i=0;i<img.pixels.length;i++)
{
  println(img.pixels[i]);  
}


2) Read the pixels by x and y index



for(int i=0;i<img.width;i++)
{
  for(int j=0;j<img.height;j++)
  {
    println(img.pixels[j*img.w idth+i]);  
  }
}

color is the object containing the color data of a pixel in Processing. It basically uses an integer to store Red, Green, Blue and Alpha value. The mathematical formula for them is :

color = alpha*255*255*255 + red*255*255 + green*255 + blue

so this color in processing = 4 244 832 000 = FFFF0000 in hexadecimal

(slightly off-topic)
However, if you try to do println(color(255,0,0)); you will get -65536. This is due to an integer in Processing contains 32bits and it can only range from -2 147 483 648 to 2 147 483 647 or [-(2^31) ~ (2*31-1)], so it uses negative numbers to represent the larger values. This method is named "Two's complement".

The lucky is that, processing or many high-level programming language did the dirty jobs already and we can use the functions easily.

alpha()
blendColor()
blue()
brightness()
color()
green()
hue()
red()
saturation()

To create a color value, we can use color() function. The below codes are identical to create this color. RGB mode is default in Processing.



 
colorMode(RGB);
color c = color(255,0,0);




 
colorMode(HSB);
color c = color(0,255,255);


 

Creating PImage

To create a PImage in processing, there are methods below:

1) Empty image PImage img = createImage(100, 100, ARGB);
2) Load from a file PImage img = loadImage("image.png");
3) Capture from canvas PImage img = get(0,0,width,height);
4) Capture from live video (e.g webcam) import processing.video.*;
 
Capture cam = new Capture(this,width,height);
cam.read();
PImage img = cam;

5) Capture from movie		 import processing.video.*;
 
Movie mov = new Movie(this, "movie.mov");
mov.read();
PImage img = mov;

note: Capture and Movie class are developed base on the PImage class, so all functions in PImage can be used on them (e.g cam.pixels[0]).
note: Actually the canvas of Processing is similar to PImage too, you can also try to access the pixels directly( get(x,y); vs img.get(x,y) ).

 


Reading and manupulate pixels in PImage

We can access the pixels and write pixels on PImage directly. However, this is a good practise for us to call loadPixels() before reading pixels[]; as well as calling updatePixels() after writing on pixels[];

loadPixels() : Loads the pixel data for the PImage into the pixels[] array.
updatePixels() : Updates the PImage with the data in the pixels[] array.

This image can be generated by below code:



PImage img;
 
void setup()
{
  size(200,200);
  img = new PImage(width,height,ARGB);
   
  //************************
  for(int i=0;i<img.pixels.length;i++)
  {
    img.pixels[i] = color(255,0,0);  
  }
  //************************
   
  img.updatePixels();
   
  image(img,0,0);
}


 

Exercise:

Try editing the above code to generate these patterns!

 

Image Processing to PImage

By exploring the manipulation of pixel data in an image, we can do some simple filter already:

source



PImage img;
 
void setup()
{
  size(320,240);
  img = loadImage("macbook.jpg");
   
  img.loadPixels();
   
  for(int i=0;i<img.pixels.length;i++)
  {
    float r = red(img.pixels[i]);
    float g = green(img.pixels[i]);
    float b = blue(img.pixels[i]);
     
    img.pixels[i] = color(r/2,g/2,b/2);
  }
   
  img.updatePixels();
   
  image(img,0,0);
}


As mentioned before, all functions in PImage are available on Movie and Capture class. So we can create realtime filter to our webcam as well.

source a

source b



import processing.video.*;
 
Capture cam;
 
void setup()
{
  size(320,240);
  cam = new Capture(this,width,height);
}
 
void draw()
{
  if(cam.available() == true)
  {
    cam.read();
    cam.loadPixels();
 
    for(int i=0;i<cam.pixels.length;i++)
    {
 float r = red(cam.pixels[i]);
 float g = green(cam.pixels[i]);
 float b = blue(cam.pixels[i]);
 float h = hue(cam.pixels[i]);
 float s = saturation(cam.pixels[i]);
 float v = brightness(cam.pixels[i]);
 
 cam.pixels[i] = color(r/2,g/2,b/2);
    }
    cam.updatePixels();
    image(cam,0,0);
  }
}


Exercise

1) Try these filters out on your webcam!


Little random values could make interesting effects too.

2) Make it interactive! (e.g interactive to mouse position)

source

 

3) Make time as one of the elements

source

Here is one of the example, Slit-scan photography. It uses a column of pixels on every captured frame and combine into a big picture.

 

Reference


Tartan Plaid from Lee Byron on Vimeo.


Have fun with image processing from mb09 on Vimeo.

Tokyo station stairwell from matt ditton on Vimeo.

Chronotopic Anamorphosis from Marginalia Project on Vimeo.

decomposing time from lukasz on Vimeo.

Turning objects 1 - vertical body from matt ditton on Vimeo.

 

 

 

 

Understanding debug error messages

Specific library did not install. Make sure libraries are put at correct path and try restarting processing.

 

Object class not found, make sure libraries are imported.

 

Null pointer exception, meaning some objects are not created / created properly. Make sure you have initizialized all the objects in the program.

 

Memory used by the program is exceed the limit. Try to enlarge the maximum memory availble for processing (in prefrence), or reduce the size and amount of your load in files. Also avoid creating too big arrays (e.g int bigArray[] = new int[100000][100000]; )