There isn't a video codec that supports alpha channel that is supported in browsers, AFAIK. So you would have to use a heavy workaround.
For <video>, in theory it would support alpha if the video codec has alpha channel information. You can change the alpha channel of the whole thing with CSS, or maybe regions of it with masks and/or filters.
<canvas> supports alpha channel, as can be seen in the demo you linked to, which paints the video on a canvas and changes the alpha channel of each pixel with ImageData.