Wednesday, February 25, 2015

2xSE: Hardware accelerated realtime pixel art upscaling

Image upscaling is nothing new. Popular algorithms such as 2xSAI and Super Eagle have been around for a decade or more but most of these algorithms run on the CPU. I wanted to write one that targeted the GPU to take advantage of hardware acceleration. Now that webGL is available, it seemed fitting write this shader in GLSL. The results thus far have been fairly encouraging. Click here to see the demo running on top of the open source JSNES emulator.

Note: use the period key on the number pad to toggle SE filtering on/off in fullscreen mode to see the difference.

Also, this goes without saying but 2xSE requires a video card with some hardware acceleration for decent performance. Software emulation of pixel shaders is not so good.

2xSE (Scaling & Extrapolation)

The algorithm works by accepting an input texture and producing an output texture that is exactly twice the size of the input. This means that for every source textel we will be producing 4 output textels. The pixel shader reads a maximum of 10 input textels for every output textel. The textels forming the diagonal line perpendicular to the pixel being written are compared. If they are the same color that color is written otherwise the color of the local textel is written.
input pattern         output
   Y  X  Y            [X][Y]
   X [Y] Y            [Y][Y]
   Y  Y  Y
         Y  <-- This extra textel has a special purpose. Keep reading...
This is enough except for In cases where the input textels form an X pattern where each stroke is a different color. This situation causes rendering artifacts so the solution is to break the tie and determine which line should get the enhancement. I chose to implement a simple luminance calculation to be the tie breaker choosing darker colors over lighter colors. This was not an arbitrary decision since most pixel sprites use a dark usually black outline and sprites should always take precedence over background images.

Fast GLSL luminance function:
float LUM(vec3 color){ 
 return (color.x+ color.x+ color.z+ color.y +color.y+ color.y) * 0.166;
The "special textel" is used only for calculating the SW output value. I use this textel as a tie breaker to flip the logic on the luminance calculation which causes the algorithm to favor lighter colors over the darker color only in this one specific case. I found that while this does reduce many artifacts it also creates some in certain cases so it was a trade off but overall I think the output looks better with this. At some point I may take it out of the shader or create an option to toggle this feature on and off.

No comments:

Post a Comment