[03-09-2022] | Selenium 4 Custom Video Recording
Selenium 4, at the time of writing, only offers processed video of test runs when running the dynamic grid configuration. Due to constraints outside of the scope of this article, I am unable to run the dynamic grid configuration as supplied by Selenium.
If not run as a dynamic grid, the video recording container provided by Selenium only checks if it can see the node's XVFB, then starts recording forever. When the containers are shut down, the video is left corrupted. Even if the FFMPEG process ends gracefully, the resultant video is as long as the node's lifetime with the test somewhere in the middle.
There are multiple issues and feature requests on SeleniumHQ’s github detailing this, with only one contributor mentioning that they ultimately wrote their own solution for a Kubernetes deployment. I was unable to find any other information about how people have handled this.
After hours of research, I was left with the conclusion that if I wanted video recording without using dynamic grid, I'd have to roll my own.
Selenium Nodes, for the purposes of this grid, are made up of an ec2 instance running a node and video container alongside each other. The node container runs XVFB on port 99, and the video container runs FFMPEG that connects to the node over the same port. As detailed in the scaling article, the video container will only ever record 1 video before being thrown away.
In order to process video, logic had to be written for the video container to:
The video container runs a loop that sends a get request to the node at 127.0.0.1:5555/status, then parses the returned JSON for a SessionId contained in the slots key. Loopback IP is used because the mapping of node to video containers is 1:1, and the node responds on port 5555 by default.
If all 3 slots (due to nodes having Chrome, Firefox, and Edge) return null SessionId, the node is not running a test. Max concurrency on a node is 1, so if 1 of 3 slots has a SessionId, a test has started and the SessionId is stored.
When a SessionId is detected, a thread that runs FFMPEG starts recording a video with a file name of the SessionId that was previously determined. While FFMPEG is running, another thread continues a loop to get the JSON output of 127.0.0.1:5555/status, and when 3 of 3 slots report that there is no SessionId, the test has ended. FFMPEG is sent a SIGTERM to cause it to end recording and stop gracefully.
Once the FFMPEG process has stopped, and is confirmed to no longer be running, a video file matching the SessionId is checked for existence. If it exists, it is uploaded to s3. After that, local MD5 is calculated and checked against the s3 MD5. If one doesn’t match the other, re-upload is attempted 3 times before exiting. After successful upload or exit, the video container sends a sigterm to itself and is destroyed.
Once both node and video containers are confirmed by the ec2 instance hosting them as not running, life begins anew after a 30 second wait period during which the instance will determine if it is up for termination or not.
HOME