Computational Bokeh

I have been challenging myself and trying to add funky shaped bokeh to photographs in post-processing. This is done using the convolution product. Click through to learn about how this works and see the results of my programing efforts.

 

---

 

When I first heard about the concept of “bokeh” with different shapes my first thoughts were “this is the convolution product!!”. In my previous post on this blog I showed you how to achieve this effect in camera, with different shaped apertures. But what if you do not have these shapes on you when you’re out shooting? Is there something you can do about this in post-processing? This is when the convolution product comes in handy.

In mathematics, the convolution product is an operation on two functions, producing a third function that is typically viewed as a modified version of one of the original functions (more details here). It is a commutative operation defined as follows:

I learnt about this operation in my course “Advanced Optics” during my physics undergraduate degree at university. And how is this useful for computing bokeh, you might ask. Well, when you convolve your photo with an arbitrary function (your aperture’s shape, for example) the result is one shape on every point of your photo. If you have bright lights in your photo they will turn into your desired shape.

Well this sounds pretty easy and straightforward, right? (hehe “nervous laugh” haha) Just get your photo and your shape, and perform this convolution magic. Technically, this should work, and I have found a very interesting article on ScratchPixel.com in which they add funky shaped boken onto a Christmas tree image using their own code written in C++.

However, I have been trying to write my own program in Matlab to do this and it is harder than I thought!!! So far, I have been only able to add bokeh to an image made out of small white dots on a black background. Like this:

Original “photograph”. Imagine that these are city lights for example hehehe

And say I want to make all my “city lights” (hehehe) turn into stars. Then I make my “aperture” to be an image that consists of a black background and a white star in the centre (seen below) and convolve the two images.

Star “aperture”

The result of this convolution magic, as predicted before, is that each dot turns into a star!! (seen below) Woooo!!! Magic!! The power of maths :)

Convolution result: Starry city lights (come on, use your imagination! hehehe)

On the bright side, I can already do different shapes!! I can make stars, hexagons, hearts, and even smiley faces :)

Hexagonal bokeh

Heart shaped bokeh

Smiley bokeh

If you look carefully you will see that these bokehs are a little bit pixelated. I have a feeling this is part of my issue but I have not been able to solve it yet. Also, the larger the images I use the less pixelated the result, I reckon this is because the imaging processing is numerical and not analytical.

Oh, and it only works in greyscale (not colour). So as you can see, I am still a long way away from achieving my end goal of adding funky shaped bokeh to a photograph, but I feel I am taking tiny steps in the right direction :) Hopefully I will be able to figure it out and share it with you at some point in the near future!

Finally, for my nerdy readers. It is faster in Matlab to do Fourier Transforms than convolution products so I used the Convolution Theorem, which states the follwing:

Where FT = Fourier Transform, * is the convolution product, and . is the normal product. So basically, I do the Fourier Transform of each image (photograph and aperture) separately, then multiply them pixel by pixel, and finally do the inverse Fourier Transform of the result. And this is equivalent to doing the convolution product.

So here is the core of my code (if only it worked with other images hahaha) (Note: in Matlab * is the normal product, not the convolution):

fft2original = fft2(original_gray);
fft2mask = fft2(mask_gray);
fft2bokeh = fft2original.*fft2mask;
bokeh = ifft2(fft2bokeh);

Those 4 lines are the faster equivalent of this single line convolution:

bokeh = conv2(original_gray, mask_gray);

I will keep you updated on how my computational bokeh attempts progress :) Wish me luck! Remember to follow me on Facebook for more updates and even nerdy jokes! hehe

Thank you for reading!!

xx Ana :)