An Insight into the HTML5 Canvas Element

During my internship at Redweb I was asked to explore what can be achieved by using cutting-edge web technologies. In particular, I was asked to experiment with the new HTML5 canvas element. To add meaning to the experiment I'd be using my research to produce an application for Redweb, one which will provide an interactive wrapper to its list of top considerations for building a website.

What is canvas and why use it?

The canvas element is interesting and worthy of focus because it enables, for the first time, direct drawing of graphics within a browser without the use for an external plugin like Flash or Java. The beauty of canvas is that it's controlled entirely via simple JavaScript code, meaning it builds on the powerful functionality JavaScript already provides and doesn't require a crazy learning curve to use.

Choosing to experiment with canvas over other new elements was simply down to it's functionality as a graphics platform, which inherently makes it a potentially interesting and rich platform to play with. It was decided that pushing the flexible canvas element would produce the most interesting results that we can use in the application.

Another deciding factor for choosing canvas was to test the animation capabilities and the possibility of it being a potential Flash replacement. Now Flash obviously has features that canvas could never emulate, however it's an exciting concept nonetheless to see exactly what could be achieved with canvas that would normally be done by reaching for Flash.

What did I decide to do with it?

After receiving the initial brief and participating in a Perspex-wall-fuelled brainstorming session it was clear that whatever I ended up making it was going to have a lot of individual components (there are over 50 of the website considerations!). There was really no entertaining of ideas relating to combining these individual considerations into a single unit.

One theme in the wall-drawing was one of nodes, as in individual dots, a theme that came back time and time again in our ideas. It was a concept that fit perfectly with the requirement of displaying the website considerations in an interesting an exciting way. If each consideration was attached to and represented by a single node, then we could create an environment in which these nodes, the considerations, could be interacted with and managed all in one place. Perfect!

The idea was later cemented while cooking at home with the project on my mind. During this exciting activity I was watching a pan of water with some cooking oil floating on top, becoming more and more intrigued with the way the oil droplets interacted and moved about on the simmering surface. As the water got hotter the individual droplets started moving around in a way that caused them to start avoiding each other as though there was a little force field around each one. It was this free-floating movement that was exactly what I wanted to replicate during the canvas experiments and integrate into the application.

Armed with this new knowledge I got straight to work.

The experiments

Before diving into the realms of a node environment I decided to explore some of the functionality that canvas provides. I was particularly interested in seeing what image support it had, and after seeing a Flash application that grabbed random colours from an image I decided to make something similar.

Manipulating images with getImageData()

It didn't take much research to discover that there is some pretty nifty image functionality baked right into canvas. One that stuck out was the getImageData() method that allowed you to grab data from any pixel in an image. Seemingly aware to the complexities on this method I jumped straight in with trying to force it under my control.

I failed.

Without going into too much detail, as it turns out to be quite a complex issue, I had basically run into one of canvas's few permission based limitations. It turns out it's quite strict on the way you can manipulate data, particularly with images. For example, you can't access pixel data from an image that doesn't come from the same "origin" (domain) as the canvas element, basically because you could bypass things like CAPTCHA by reading the pixel data. More information can be found on the canvas security documentation. Needless to say that I conquered the issue for my needs by moving the experimental files to a proper server (www.[…].com) rather than running it off the computer file-system (file://…).

I didn't pursue the idea much further than a technical demo but you can see the final experiment on my Vimeo profile. The end result turned out to be particularly interesting as a colour scheme picker tool and proved a satisfying introduction to canvas.

Getting started with nodes and physics

Drawing nodes on canvas isn't particularly hard, it requires a few lines of script which in essence create a circle and then fill it in with a colour of your choice. It really starts to get interesting when you want to animate those nodes and even more so when those nodes need to become interactive. Both features which I needed to implement.

The underlying solution to both issues is an object oriented programming (OOP) approach and creating each node as an individual object based on a Node class (a cookie-cutter template of sorts). In itself this doesn't achieve much but when you then store a reference to each of these node objects within an array you've created a basic node control system. The reason the array brings the whole system together is because it allows you to keep a single reference to all the nodes and loop through each one and manipulate it at your leisure. If you loop through those nodes at a set interval, say every 30 milliseconds (about 30 fps), then you've created a basic animation system. Take that Flash! By building on this system of animating individual node objects inside a collection array I was able to start experimenting with the canvas again.

Movement was a major milestone in the project and proved to be one of the most complicated features to pull off. It was relatively trivial to create a basic script that moved each node X pixels in a certain direction on each frame of the animation, but making each node move on its own accord in a natural way seemed impossible. It probably didn't help that I knew the solution lied in mathematics, a subject area I didn't particularly pride myself in.

Regardless of my fear of formulas I pressed ahead and started poking around with basic physics, particularly anything related to velocity and motion. It turns out a guy by the name of Sir Isaac Newton created a perfect set of rules called Newton's laws of motion. Combining this with other information I uncovered about creating motion with script I stumbled across a gem of a technique called Verlet integration, a formula for calculating the trajectory of an object. Without going into the nitty gritty math, Wikipedia does that better than I can, if it's good enough for game programming then it's good enough for me.

With this fantastically complex formula I could apply a force to any node and watch as it moved itself, dynamically. I was one step closer to achieving the simmering water effect I was striving for.

The final piece in that puzzle was to make each node move completely at random. This wasn't so hard and involved giving each node a random force on every frame of the animation; this basically pushes each node in a different direction and creates an illusion of natural movement. It's not the best solution but it's certainly good enough and looks believable.

After adding some further script to detect which nodes are closest to another one, via basic trigonometry, I was able to implement a system that allowed them to interact with each other. The first element of this system that I introduced was a way for the nodes to take on the colour of another node if they collided.

I now had a working concept of an animated, natural, node system. You can see the video of this stage on my Vimeo account.

A note on performance

With over 50 nodes floating around and running quite substantial code every 30 milliseconds, I was starting to become aware of the performance limitations of canvas with current JavaScript engines.

The problem lay with the way the script has been written that detects nodes nearest to another node. At this point the script looped through every node (over 50 of them) and on each iteration it then looped through every node one further time to discover where other nodes are in relation to the current one. That is around 2500 loops if you have a 50 node system. Couple this with the fact it's running all these loops every 30 milliseconds and you have a pretty resource intensive process going on. This process wasn't proving to be a crippling problem but it certainly made me very aware of the impact certain script will have on performance. If I increased the number of nodes to something ridiculous, say 100, then you would see a very jerky and unsatisfying animation.

On the bright side these issues are only a problem on older computers with small processing power, and older browsers that don't utilise the next-gen JavaScript engines like V8 (Chrome) and Tracemonkey (Firefox); browsers that haven't been designed to work particularly well with canvas anyway. All newer computers and browsers I tested in proved to handle the script without a sweat and were extremely smooth. An obvious sign that the future is bright for JavaScript based applications and the canvas element.

Regardless, by taking these issues into consideration I was able to press on with the system and utilise good programming practice to optimise future script and avoid potential performance hits.

Tweaking the system

The system, as it currently stood, was basic and the node collision system was in particular need of refinement.

At the moment the nodes didn't know if they were actually touching each other and they were just using a static distance value to decide what was close and what wasn't. To change this the system was tweaked so that the distance value, used to work out what was classed as ‘touching' the node, was made dynamic and directly related to the radius (size) of the node.

This was a simple tweak but the effects were interesting as the colour swapping script was now only run at the point one node's edge touched another. See this video for an example.

To further enhance the colour swapping script I changed the algorithm to run whenever another node is within a certain perimeter around the chosen node. What this allowed me to do was make the colour of that intruding node shift ever so slightly the longer it stays close to the chosen node. If it stays in the perimeter long enough, without touching, then it'll eventually change to the colour of the chosen node. The effect it produced is one that is visually pleasing and pretty interesting to watch as it's the first attempt at making the nodes really interact with one another. If you watch this video you'll see the hypnotic effect.

Visualising all the wonderful data

One subject area I was quite interested in exploring was data visualisation, and through the canvas system I had so far I felt I was ready to give it a go.

The great thing about the way the system has been set up is that it has a multitude of data about each node to choose from, including position, velocity, colour, size, mass, friction. It was just a case of thinking up a way to visualise some of it. As the nodes are in constant motion I decided to work on a method of visualising the velocity data in an interesting way, but also more as a stepping stone into data visualisation.

After some sketching of ideas I settled on using bars to display the velocity as it can not only show the size of the velocity, but by mounting a bar on each axis of the node it can show the direction as well. Implementing this functionality was fairly simple with the node system in place as it meant I could just adapt the current Node class (the template) to allow for data visualisations. In essence I added script that grabbed the velocity amounts and drew bars of that size on the node on each frame of the animation. When animated the bars fluctuate because of the changing velocity and this produces an interesting effect, as seen on this video.

Pulling it all together

The purpose of all these experiments was not only to test out canvas, but also to ultimately produce a usable application for Redweb's website considerations. With this is mind I started to knuckle down and work out how to use everything that's been created so far to compile into a single application with a purpose.

From the beginning, interactivity was one of the main requirements of the final application. It was pretty obvious how I was going to achieve this in a way that linked to the website considerations, and that was by making the nodes clickable. The reason this method was so obvious was that it had always been a concept floating around through the development of the experiments. The general idea was that you'd be able to click on a node as it's floating around and it would then expand and move to the centre of the browser window. From there the nodes left on the canvas would fall into a circular formation around the central node, creating a sort of orbiting ring. By then displaying the consideration related to the chosen node I had a system which was interactive and also met the basic need of displaying the considerations in an interesting way.

Creating the final features to the system was relatively straight forward now I knew how I wanted it to work. The animation system was tweaked so it tracked when a node was clicked and if so, would enlarge the node, move it towards the centre and display the attached consideration. From there I added some functionality which looped through the remaining nodes, worked out their position on the outer circle, and then proceeded to move them into position frame by frame. The final product was a lot more complex but the basic theory behind the script is the same.

You can see a video of the application progress here, and a video of the finished application here.

Results and conclusion

Working with canvas has been incredibly interesting and a lot of fun. I've not only learnt a lot about its functionality and possibilities, but I've also picked up a lot of new JavaScript and programming techniques.

The results of the experiments alone have proven to me that canvas is a very capable platform and although it has performance issues relating to JavaScript, it's certainly worth considering for drawing and interactive application needs. Flash certainly won't be the first tool on my mind in the future!

In the 3 weeks working on this project I've only been able to scratch the surface of what other treasure canvas hides below the surface. It's certainly safe to say that I've got plenty of ideas and wishes for future development with the new element, though.

I'm looking forward to canvas becoming a fully supported feature in all browsers when HTML5 is finalised. It's going to allow for some interesting developments in web design.