How best to stream video to the cloud using AXIS devices
There are many protocols and possibilities, we’ve evaluated each to save you time, and share examples of how to send video to AWS from your AXIS devices for your video analytics
What to look for in your streaming method?
A lot of computer vision and machine learning engineers these days need a way to get their videos to the cloud for analysis. AXIS devices provide a great platform for this, but they provide a dizzying range of ways to get your video to the cloud, so which should you choose and what is the most production-worthy solution?
Like many companies, we had a use case whereby we needed to send video to the cloud which meets the following:
- Secure (encrypted)
- Cost efficient
- Easy to scale to hundreds of cameras
- Uninterrupted stream (no gaps in the transmitted video) ️
- Robust to brief network outages (stores video locally during outages)
How about hosting your own file server?
When considering which option to use, a key consideration is whether you need to manage your own file server to receive the files, e.g. via a cheap EC2 instance. Whilst an EC2 instance hosting a file server to receive videos may seem at first a sensible option, disadvantages include:
- The increased cost of maintenance (e.g. security patches and risk of it being hacked)
- The fact that cheaper EC2 instances may have insufficient networking capability for your workload [4]; a little advertised feature of EC2 is that cheaper instances often have poor networking performance.
- As you scale your number of videos, you’d need to manually maintain an increased number or scale of EC2 instances, and this has a large labour and maintenance cost.
That’s why we in Scoville are big fans of the Amazon Kinesis Video Streams service — a serverless solution whereby AWS seamlessly manages the scaling and security on your behalf — helping to reduce your Total Cost of Ownership significantly. But let’s take a step back and look at what options are available.
What’s the best protocol to send video to the cloud on AXIS devices?
We’ve compared protocols for you and provided a repo with setup instructions to help you try them out.
To explain the summary table, we’ll break down each option below:
1: File Transfer Protocol (FTP): In newer Axis OS firmware, FTP has been disabled by default as it is an insecure protocol [5] and it’s unencrypted so insecure. You’ll need to maintain and scale your own file servers if you go with this approach, which will quickly become expensive and impractical to maintain as you increase the number of video feeds.
2: Secure File Transfer Protocol (SFTP): Same problems as above, but with the benefit of encryption and the big disadvantage that this form of encryption is too slow to transmit HD video [1] [2]. Another consideration is that it only supports passphrases, not keypairs for authentication, which is less conventional for SSH-based authentication [3].
3 & 4: Server Message Block (SMB): Some older AXIS firmware does not support SMB 3.0, and SMB 2.0 is unencrypted to insecure. SMB protocol requires a file server (e.g. Samba server) which can receive the files, meaning much like FTP it will quickly become expensive and impractical to maintain as you increase the number of video feeds.
5. HTTP: This is an insecure protocol as it is unencrypted. If you send it to a manually hosted web server (e.g. running NGINX) you’ll have scaling and affordability issues as you increase the number of video feeds — but it should be noted that there are affordable and scalable serverless solutions for HTTP like AWS API Gateway. If you use this approach, there is often going to be a limit on the payload side (e.g. 10MB for API Gateway) which will mean you need to work with very small video clips (see next section as to why that’s very problematic).
6: HTTPS: This adds encryption to HTTP, and whilst it doesn’t suffer from the issue that SFTP has related to slow streaming rates, it does rely on sending video clips which can be problematic as explained in the next section.
7: RTSP “Pull”: It’s possible to connect to the camera from an external server and download video via the RTSP protocol. This can provide in theory an uninterrupted stream when using a web server pulling the stream. We tried pulling via `ffmpeg -rtsp_transport` and intermittently (about once a day) we encountered issues with the ffmpeg software freezing for reasons we could not diagnose. This approach should work theoretically with the right tooling, however, this approach is not without disadvantages:
A) Security: By default, RTSP is unencrypted, so to make it secure you’d need to connect to the camera via an encrypted method like SSH or HTTPS.
B) Scaling: As with the other approaches, you’d need to scale and maintain your own web servers to pull the video, which will quickly become expensive and impractical to maintain as you increase the number of video feeds.
C) Networking: As this is a pull method, it means that you cannot simply set up the device and “forget” — you instead need to establish some connection from outside of the network to inside your network with the camera — which can be tricky and expensive to maintain at scale — for example setting up SSH connections or a VPN connection between your server and every camera
8: RTSP “Push”: AXIS devices do not natively run an RTMP server on them and so natively cannot “push” video from the device to an external server.
9: KVstreamer: KVstreamer is software that you install on your AXIS device, and it enables it to send an uninterrupted stream to Amazon Kinesis Video Streams service. Unlike all the prior methods, it is even robust to short network outages, meaning if there is a brief network outage then when the internet comes back online, it will use its offline cache to send the video to the cloud meaning nothing is lost. As Amazon Kinesis Video Streams is a fully serverless AWS-managed service, it means you do not need to worry about maintaining and scaling file servers as your number of video streams grows.
Can I get an uninterrupted stream using these approaches?
The short answer is yes with KVstreamer and probably not with the other methods, in our experience.
Whilst there are some protocols suitable for securely sending short video clips from AXIS devices to the cloud, our experiments suggest KVstreamer is the best solution to securely stream uninterrupted video to the cloud.
References
For details of experiments performed for this blog article, please see https://github.com/scoville/camera2aws-protocol-comparison
[1] SolarWinds support, “SFTP transfer slower than FTP”, viewed on 8th June 2023, <https://support.solarwinds.com/SuccessCenter/s/article/SFTP-transfer-slower-than-FTP?language=en_US>
[2] AXIS User Manual, “SFTP is a more secure method than FTP but file transfer might be slower, especially for large files such as high resolution video” (accessed 2023),<https://www.axis.com/dam/public/87/bf/8d/axis-q1942-e--user-manual-prior-to-fw-720-en-US-107779.pdf>
[3] AXIS VAPIX API, “Only supports passphrase not keypairs for authentication” (accessed 2023), <https://www.axis.com/vapix-library/subjects/t10175981/section/t10008227/display>
[4] Andreas Wittigm, “EC2 Network Performance Cheat Sheet” (2018), <https://cloudonaut.io/ec2-network-performance-cheat-sheet/>
[5] AXIS FAQ, “How do I enable the FTP Server on a AXIS product?” (accessed 2023), <https://www.axis.com/support/faq/FAQ116316>
[6] We encountered “GDBus.Error:com.axis.Streamer.Buffer.Error.Retry: Request can not be serviced at this time, retry the operation” periodically during this test which we believe is related to a lack of robustness to timeouts in the firmware when contacting SMB servers with high latency (such as in a different country)
[7] Benjamin M. Lowe, “Comparing supported protocols on AXIS devices to send video streams over the network to AWS”, (2023), <https://github.com/scoville/camera2aws-protocol-comparison/