A portrait clip opens fine on the watch page. Tall, upright, nothing to see. Then you scroll back to the list and its thumbnail is a wind turbine lying on its side. I build small internal tools for a content team, and this is the kind of bug that lands on my desk as a screenshot with three question marks.
It took me a while to accept that the video was never upright in the first place. A lot of portrait video is stored with the pixels lying down, as an ordinary landscape picture, and the file carries a separate note that says "turn this 90 degrees when you play it". I think of it as a sticky note on the box. A player reads the note and turns the picture. Anything that opens the box without looking at the lid gets the picture the way it was packed.
To find out who reads the note, I took an example clip from Wikimedia Commons, a wind turbine against a blue sky. I painted a red block in its top-left corner and a green one in the top-right to mark which way is up. Then I had ffmpeg 7.1 store it sideways as 1920×1080 H.264 with the note attached, which ffmpeg reports as rotation of -90.00 degrees. So this is a lab sample, not footage straight off a phone, and I didn't test HEVC, iPhone Safari or Android. I grabbed a frame from it five different ways in Chromium 149, Chrome 154, Firefox 151 and WebKit 26.5 on macOS.
Read each row left to right and follow the red block. Top-left means the frame is upright, bottom-left means it is lying down. The turbine footage is by Grendelkhan on Wikimedia Commons, CC BY-SA 4.0.
Two of the five never let me down. Plain <video> playback was upright in all four browsers and reported videoWidth × videoHeight as 1080×1920. canvas.drawImage(video), the old trick of painting the current frame onto a canvas, was upright in all four as well. The trouble starts with the two newer shortcuts.
const bmp = await createImageBitmap(video); // the current frame as a ready-to-draw bitmap const vf = new VideoFrame(video); // the current frame as a WebCodecs object console.log(bmp.width, bmp.height, vf.displayWidth, vf.displayHeight, vf.rotation); In Chrome 154 and WebKit 26.5 both lines gave me an upright 1080×1920 frame. In Firefox 151 both came back as 1920×1080 with the turbine on its side, and the VideoFrame had no rotation property at all, so there was nothing to check against. Chromium 149 found a third answer. Its createImageBitmap returned a 1920×1080 bitmap whose picture was the right way up but squashed: my 160×160 corner blocks came out at roughly 284×90. Chrome 154 runs the same line correctly. I haven't gone looking for why the two builds differ.
The fifth way is the one where every browser agrees, and they agree on the wrong answer. If you unpack the MP4 yourself (I used mp4box.js 0.5.2) and feed the compressed samples to VideoDecoder, the browser's raw decoder, the frames come out sideways in all four. In the two Chromium builds the decoded frame even says rotation is 0. That is when the sticky note finally clicked for me. The note is not inside the video stream. It sits in the container, in a field of the track header called the matrix, and the decoder only ever sees the stream. mp4box will show it to you if you ask.
const t = info.videoTracks[0]; const m = t.matrix || file.getTrackById(t.id).tkhd.matrix; console.log(t.track_width, t.track_height, Array.from(m)); // 1920 1080 [0, 65536, 0, -65536, 0, 0, 0, 0, 1073741824] Even the width and height in there are the packed size, not what a viewer sees. So when I do the unpacking myself, the rotation is my job too. I assume <video> gets it right because the browser's own unpacker reads the matrix and passes it along, but that is a guess from behaviour, not from reading source.
The container matters in the other direction too. I moved the same sideways pixels and the same note into a VP9 WebM with ffmpeg's -noautorotate, and ffmpeg still read -90 from the file. Chromium 149 and Chrome 154 played it upright at 1080×1920. Firefox 151 and WebKit 26.5 played it lying down at 1920×1080. It was the only time plain playback of a file that still carried its note went wrong for me.
What I do now is boring. For a thumbnail from a playing video I stay with drawImage. For files I control, I get rid of the note by baking the turn into the pixels once. ffmpeg's default transcode did that for my sample: 1080×1920 out, no rotation tag. For the no-install route I used ImgIng (https://imging.ai/) in Chromium 149 on its size-first preset, and the samples tagged 90, 270 and 180 degrees each came out as 608×1080 with upright pixels and no rotation note.
The 180 one still bothers me. In every cell that failed, that sample came out upside down with its width and height unchanged, so a "wider than tall" check catches nothing. My corner blocks catch it, but only on files I made myself.
