The testing guidelines address the following areas to help you test smarter, and more efficiently:
Most Axis IP camera models are categorized as either M, P or Q line. These categories indicate differences in areas like power consumption, level of performance, RAM, etc., things that will affect number of streams, FPS, etc.
The fact that these categories of cameras differ in specifications also means these groups will differ in the required testing. Consider this in your test strategy so that you don’t expect the same results from different cameras.
In an Axis network video product, the AXIS OS functions as the operating system, being the software that manages all aspects of the product’s performance. That includes, for example:
AXIS OS is customized for each product, to support both the imaging hardware and any product specific features. However, AXIS OS is developed on and released from a single code branch, ensuring that different product models will have similar or same behavior if they have the same AXIS OS version. Before a new product is released, the stability of the AXIS OS is ensured through extensive testing of the product’s features.
After release, AXIS OS updates are scheduled to add features and/or to fix bugs. These updates are also tested extensively, both to ensure continued stability and to maintain backward compatibility.
The product name reveals various product specifications in a standardized way. To learn more about the naming convention for Axis devices, run the online course.
Axis offers two separate AXIS OS tracks, Active and Long-Term Support (LTS), which both have their own release intervals. These tracks also differ in the number of changes that can be expected in each release, so the test strategy for each type should be different.
For some older camera models and in exceptional cases, there may be product specific AXIS OS versions, but this is the exception. Most AXIS OS releases are either on the Active track or the LTS track.
Visit the AXIS OS portal to read more about the different tracks.
With these insights on Axis’ different AXIS OS tracks, and with consideration to which track supports your business objectives, it’s time to look at the test strategy. Our main purpose with these guidelines and recommendations is to help you test smarter, so let's see how you can use the above knowledge to improve your interoperability testing.
Instead of “testing everything on every camera and every AXIS OS release” it is possible to adjust the testing based on risk. Here we cover two dimensions of risk:
As a rule of thumb, similar names will mean that the cameras are similar. But cameras can also be similar from a video management system point of view, although the names are radically different. For interoperability testing, the most important grouping parameters are:
Depending on the risk accepted, the whole Axis portfolio of roughly 100-200 cameras could be represented by 3-10 cameras for testing purposes.
The AXIS OS versions on the Active track and the AXIS OS versions on the LTS track are quite different in the number of changes that can be expected in each release, so the test strategy for each type should be different.
All AXIS OS releases that are on the Active track (x.y0, e.g. 9.10) have been developed and tested by Axis on a common code base. Therefore, they can be grouped with high confidence. The changes on the Active track are greater than on an LTS track, so risk due to code changes is higher than for LTS.
Recommended test strategy:
Within each LTS track, there are typically very small changes and Axis will do everything to keep new LTS releases as backward compatible as possible. All AXIS OS releases that have the same LTS version have been developed and tested together. For this reason, the risk due to code changes is very low and very aggressive grouping can be done.
Recommended test strategy:
For some older camera models and in exceptional cases, there may be product specific AXIS OS versions. Any product specific AXIS OS version (x.y5, e.g. 8.55) is specifically built for that product. Although it is based on the common AXIS OS platform, it has been developed and tested independently and as such it carries the greatest risk.
Recommended test strategy:
If the number of new cameras and new AXIS OS versions from Axis is a challenge, then a good solution can be to primarily focus on supporting the LTS tracks. This is the track that requires by far the least amount of work to test and to maintain, as the changes from the Axis side are small. Axis has a focus on backward compatibility on the LTS tracks, which means that each release on an LTS track should be fully backwards compatible with previous releases on that track, making it safe and easy to keep AXIS OS updated.
This means that two substantial benefits of fully supporting an LTS track are:
However, it should also be noted that new cameras typically are introduced on the Active track and there are other benefits of also following Active, or at least keeping an eye on the Active track.
Following the Active track has benefits, also on the testing:
Focusing on LTS and not Active track releases will mean less test effort and therefore can be a good choice. However, stacking up a large amount of changes will make compatibility issues harder to track down. What you save in testing may result in greater surprises later on.
If you follow the Active track, you will continuously upgrade your device integration support and possible issues will be handled regularly. This means that any such issue should not be so widespread before you have a chance to identify and correct it, i.e., you get a quick feedback loop by following the Active track. If you don’t, potential issues might get widespread usage before you notice it and that makes the deployment of fixes much more of a challenge. The speed with which possible bugs and compatibility issues are reported back to Axis will minimize how much impact the issue will have on your customers.
New camera models are primarily released on the Active track. These cameras can be used as a part of a test group and thus make the testing more efficient. E.g. if the new model P1468-LE is released with 10.11, then this new camera can be used to represent a group of other cameras on 10.11. In this way, any testing done to integrate the new camera with 10.11 can be counted towards the testing of 10.11 in general. Following the Active track will make it easier to integrate new cameras.
If a camera was first released with a product specific AXIS OS version, e.g. 8.55, and now there exists a later Active or LTS version, then the original AXIS OS version should be ignored, and a newer AXIS OS release should be used for integration.
The selection of devices to use when testing new AXIS OS releases should be risk based. Axis can provide general recommendations, but you will have to ask yourself the following questions:
The following list is an example that can be used for testing after track AXIS OS version 10.12 and later releases. The devices have been selected based on both features and hardware platforms.
Priority | Camera name | Chip platform | Special features |
1 | AXIS Q6075 PTZ Network Camera | ARTPEC-7 | PTZ |
2 | AXIS Q1656-LE Box Camera | ARTPEC-8 | Latest chipset |
3 | AXIS P1375 Network Camera | ARTPEC-7 | Popular model |
4 | AXIS M3077-PLVE Network Camera | ARTPEC-7 | Fisheye |
5 | AXIS FA54 Main Unit | ARTPEC-6 | Modular |
6 | AXIS I8016-LVE Network Video Intercom | ARTPEC-7 | Intercom |
7 | AXIS M4216-V/LV Dome Camera | CV25 | |
Others to consider (in no particular order) | |||
AXIS D2110-VE Security Radar | ARTPEC-7 | Security radar | |
AXIS C1310-E Network Horn Speaker | 6ULL | Speaker |
Please note that this list is only an example. Other devices can be more relevant based on region, vertical, VMS feature set, etc. The important part is selecting a set of devices that is a good representation of the differences in the Axis portfolio and a relevant sample for your context.
Log in to your technology integration partner account to gain access to the latest AXIS OS beta and test versions on the partner web. Read more about the Axis Technology Integration Partner Program.