(This article was originally published at dahtah, and syndicated at StatsBlogs.)
Superpixels are used in image segmentation as a pre-processing step. Instead of segmenting pixels directly, we first group similar pixels into “super-pixels”, which can then be processed further (and more cheaply).
(image from Wikimedia)
The current version of imager doesn’t implement them, but it turns out that SLIC superpixels are particularly easy to implement. SLIC is essentially k-means applied to pixels, with some bells and whistles.
We could use k-means to segment images based on colour alone. To get good results on colour segmentation the CIELAB colour space is appropriate, because it tries to be perceptually uniform.
library(tidyverse)
library(imager)
im <- load.image("https://upload.wikimedia.org/wikipedia/commons/thumb/f/fd/Aster_Tataricus.JPG/1024px-Aster_Tataricus.JPG")
#Convert to CIELAB colour space, then create a data.frame with three colour channels as columns
d <- sRGBtoLab(im) %>% as.data.frame(wide="c")%>%
dplyr::select(-x,-y)
#Run k-means with 2 centers
km <- kmeans(d,2)
#Turn cluster index into an image
seg <- as.cimg(km$cluster,dim=c(dim(im)[1:2],1,1))
plot(im,axes=FALSE)
highlight(seg==1)
We mostly manage to separate the petals from the rest, with a few errors here and there.
SLIC does pretty much the same thing, except we (a) use many more centers and (b) we add pixel coordinates as features in the clustering. The latter ensures that only adjacent pixels get grouped together.
The code below implements SLIC. It’s mostly straightforward:
#Compute SLIC superpixels
#im: input image
#nS: number of superpixels
#ratio: determines compactness of superpixels.
#low values will result in pixels with weird shapes
#... further arguments passed to kmeans
slic <- function(im,nS,compactness=1,...)
{
#If image is in colour, convert to CIELAB
if (spectrum(im) ==3) im <- sRGBtoLab(im)
#The pixel coordinates vary over 1...width(im) and 1...height(im)
#Pixel values can be over a widely different range
#We need our features to have similar scales, so
#we compute relative scales of spatial dimensions to colour dimensions
sc.spat <- (dim(im)[1:2]*.28) %>% max #Scale of spatial dimensions
sc.col <- imsplit(im,"c") %>% map_dbl(sd) %>% max
#Scaling ratio for pixel values
rat <- (sc.spat/sc.col)/(compactness*10)
X <- as.data.frame(im*rat,wide="c") %>% as.matrix
#Generate initial centers from a grid
ind <- round(seq(1,nPix(im)/spectrum(im),l=nS))
#Run k-means
km <- kmeans(X,X[ind,],...)
#Return segmentation as image (pixel values index cluster)
seg <- as.cimg(km$cluster,dim=c(dim(im)[1:2],1,1))
#Superpixel image: each pixel is given the colour of the superpixel it belongs to
sp <- map(1:spectrum(im),~ km$centers[km$cluster,2+.]) %>% do.call(c,.) %>% as.cimg(dim=dim(im))
#Correct for ratio
sp <- sp/rat
if (spectrum(im)==3)
{
#Convert back to RGB
sp <- LabtosRGB(sp)
}
list(km=km,seg=seg,sp=sp)
}
Use it as follows:
#400 superpixels out <- slic(im,400) #Superpixels plot(out$sp,axes=FALSE) #Segmentation plot(out$seg,axes=FALSE) #Show segmentation on original image (im*add.colour(abs(imlap(out$seg)) == 0)) %>% plot(axes=FALSE)
The next step is to segment the superpixels but I’ll keep that for another time.
Update: as reported on Github, some Windows users might run into a problem when loading the test image directly from a URL. The solution is to add Wikimedia to your list of trusted sites, see here</a.
Please comment on the article here: dahtah
