Benchmarks

A compliance analysis compares the target system to a reference (the benchmark) which is a set of security rules to assess the compliance or non-compliance of the system.

A benchmark is defined using the SCAP standard that allows organizations to automate how they monitor vulnerabilities and ensure they are compliant with security policies.

Structure of a benchmark

A benchmark includes an XCCDF file that describes the compliance rules and the tests to evaluate them. These tests can be written in OVAL or SCE format.

Cyberwatch supports two benchmark formats:

a ZIP file containing at least an XCCDF file and possibly other files (SCE or OVAL scripts) whose relative paths have been declared in the XCCDF
a file in DataStream format

Minimalist example of XCCDF file

Example XCCDF file extracted from the CIS benchmark for Ubuntu Linux 22.04 LTS that defines a benchmark containing only one compliance rule:

<xccdf:Benchmark id="xccdf_org.cisecurity.benchmarks_benchmark_1.0.0_CIS_Ubuntu_Linux_22.04_LTS_Benchmark" xmlns:xccdf="http://checklists.nist.gov/xccdf/1.2" xmlns="http://checklists.nist.gov/xccdf/1.2" >
  <xccdf:title>CIS Ubuntu Linux 22.04 LTS Benchmark</xccdf:title>
  <xccdf:platform idref="cpe:2.3:o:canonical:ubuntu_linux:22.04:*:*:*:lts:*:*:*"/>
  <xccdf:Rule id="xccdf_org.cisecurity.benchmarks_rule_1.1.5.3_Ensure_noexec_option_set_on_varlog_partition" role="full" selected="false" weight="1.0">
    <xccdf:title xml:lang="en">Ensure noexec option set on /var/log partition</xccdf:title>
    <xccdf:description xml:lang="en">
      <xhtml:p><xhtml:span class="inline_block">noexec</xhtml:span> mount option specifies that the filesystem cannot contain executable binaries.</xhtml:p>
    </xccdf:description>
    <xccdf:rationale xml:lang="en">
      <xhtml:p> Since the <xhtml:span class="inline_block">/var/log</xhtml:span> filesystem is only intended for log files, set this option to ensure that users cannot run executable binaries from <xhtml:span class="inline_block">/var/log</xhtml:span>.</xhtml:p>
    </xccdf:rationale>
    <xccdf:fixtext xml:lang="en">
      <xhtml:div>
        <xhtml:p><xhtml:strong>IF</xhtml:strong> the <xhtml:span class="inline_block">/var/log</xhtml:span> partition exists, edit the <xhtml:span class="inline_block">/etc/fstab</xhtml:span> file and add <xhtml:span class="inline_block">noexec</xhtml:span> to the fourth field (mounting options) for the <xhtml:span class="inline_block">/var/log</xhtml:span> partition.</xhtml:p>
        <xhtml:p>Run the following command to remount <xhtml:span class="inline_block">/var/log</xhtml:span> with the configured options:    </xhtml:p>
        <xhtml:code class="code_block"># mount -o remount /var/log</xhtml:code>
      </xhtml:div>
    </xccdf:fixtext>
    <xccdf:complex-check operator="AND">
      <check system="http://oval.mitre.org/XMLSchema/oval-definitions-5">
        <check-content-ref href="CIS_Ubuntu_Linux_22.04_LTS_Benchmark_v1.0.0-oval.xml" name="oval:org.cisecurity.benchmarks.canonical_ubuntu_linux_22:def:2956581"/>
      </check>
    </xccdf:complex-check>
  </xccdf:Rule>
</xccdf:Benchmark>

A few elements to note in this document:

The benchmark identifier defined by the id attribute on the <xccdf:Benchmark> tag which serves as a reference for Cyberwatch
The target platform is defined in CPE format by the idref attribute on the <xccdf:platform> tag. It is also possible to declare one or more platforms at the rule level if you want to restrict its application
In the definition of the rule <xccdf:Rule>, the content of the tag <xccdf:complex-check> is used to list the tests to run and the relative paths to access the files where they are defined

In this example, the only rule references a test in OVAL format, the benchmark will therefore contain this XCCDF file and the declared OVAL file. It can then be distributed in ZIP format or in DataStream format.

On the other hand, if the rule had referred to a script in SCE format, by declaring for example:

<check system="http://open-scap.org/page/SCE">
  <check-import import-name="stdout"/>
  <check-content-ref href="sce/ensure_no_exec_option.sh"/>
</check>

The benchmark would contain the XCCDF file and a sce directory containing the Shell script ensure_no_exec_option.sh. All files should be compressed into a ZIP file in order to be added to Cyberwatch, as described below.

The tree structure of the files in the ZIP file must imperatively respect the relative paths declared in the XCCDF file.

Add a benchmark

Go to Settings > Benchmarks
Click « Add »
Select the benchmark to add, in ZIP or DataStream format
Click « Import »

During import, if the benchmark identifier already exists in Cyberwatch, the benchmark is updated. Otherwise, a new benchmark is created.

It is possible to import new benchmarks only for the platforms supported by the compliance management module of Cyberwatch. It is therefore possible to import a benchmark for other versions of operating systems such as macOS or Ubuntu 12.04 for example, but not to import a benchmark to check a software configuration or network equipment for example.

List of supported platforms

The following table lists the supported platforms and the benchmarks already installed:

Operating system	CPE of the platform	Installed benchmarks
AlmaLinux	cpe:/o:almalinux:almalinux:8, …	CIS Alma Linux OS 8 Benchmark CIS AlmaLinux OS 9 Benchmark
Amazon Linux 2	cpe:/o:amazon:linux_2	CIS Amazon Linux 2 Benchmark
Apple macOS	cpe:/o:apple:mac_os_x:10.15.0, …	CIS Apple macOS 10.15 Catalina Benchmark CIS Apple macOS 11.0 Big Sur Benchmark CIS Apple macOS 12.0 Monterey Benchmark CIS Apple macOS 13.0 Ventura Benchmark
CentOS Linux	cpe:/o:centos:centos:6, …	CIS CentOS Linux 6 Benchmark CIS CentOS Linux 7 Benchmark
Debian	cpe:/o:debian:debian_linux:9.0, …	CIS Debian Linux 9 Benchmark CIS Debian Linux 10 Benchmark CIS Debian Linux 11 Benchmark
Fedora Linux	cpe:/o:fedora:linux:28	CIS Fedora 28 Family Linux Benchmark
Microsoft Windows 8	cpe:/o:microsoft:windows_8	CIS Microsoft Windows 8 Benchmark
Microsoft Windows 10	cpe:/o:microsoft:windows_10	CIS Microsoft Windows 10 Enterprise Benchmark CIS Microsoft Intune for Windows 10 Benchmark
Microsoft Windows 11	cpe:/o:microsoft:windows_11	CIS Microsoft Windows 11 Enterprise Benchmark CIS Microsoft Intune for Windows 11 Benchmark
Microsoft Windows Server 2008	cpe:/o:microsoft:windows_server_2008, cpe:/o:microsoft:windows_server_2008:r2	CIS Microsoft Windows Server 2008 (non-R2) Benchmark CIS Microsoft Windows Server 2008 R2 Benchmark
Microsoft Windows Server 2012	cpe:/o:microsoft:windows_server_2012, cpe:/o:microsoft:windows_server_2012:r2	CIS Microsoft Windows Server 2012 (non-R2) Benchmark CIS Microsoft Windows Server 2012 R2 Benchmark
Microsoft Windows Server 2016	cpe:/o:microsoft:windows_server_2016	CIS Microsoft Windows Server 2016 Benchmark
Microsoft Windows Server 2019	cpe:/o:microsoft:windows_server_2019	CIS Microsoft Windows Server 2019 Benchmark
Microsoft Windows Server 2022	cpe:/o:microsoft:windows_server:2022	CIS Microsoft Windows Server 2022 Benchmark
Oracle Linux	cpe:/o:oracle:linux:6, …	CIS Oracle Linux 6 Benchmark CIS Oracle Linux 7 Benchmark CIS Oracle Linux 8 Benchmark CIS Oracle Linux 9 Benchmark
Red Hat Enterprise Linux	cpe:/o:redhat:redhat_enterprise_linux:6, …	CIS Red Hat Enterprise Linux 6 Benchmark CIS Red Hat Enterprise Linux 7 Benchmark CIS Red Hat Enterprise Linux 8 Benchmark CIS Red Hat Enterprise Linux 9 Benchmark
Rocky Linux	cpe:/o:rocky:rocky_linux:8, …	CIS Rocky Linux 8 Benchmark CIS Rocky Linux 9 Benchmark
SUSE Linux Enterprise Server OpenSUSE Leap	cpe:/o:suse:suse_linux_enterprise_server:12, …	CIS SUSE Linux Enterprise 12 Benchmark CIS SUSE Linux Enterprise 15 Benchmark
Ubuntu	cpe:/o:canonical:ubuntu_linux:16.04, …	CIS Ubuntu Linux 14.04 LTS Benchmark CIS Ubuntu Linux 16.04 LTS Benchmark CIS Ubuntu Linux 18.04 LTS Benchmark CIS Ubuntu Linux 20.04 LTS Benchmark CIS Ubuntu Linux 22.04 LTS Benchmark
Linux kernel	cpe:/o:linux:linux_kernel:*	CIS Docker Benchmark

Supported standards

The following SCAP components, with their XML namespace, are supported:

DataStream 1.2 (http://scap.nist.gov/schema/scap/source/1.2)
XCCDF 1.2 (http://checklists.nist.gov/xccdf/1.2)
OVAL 5 (http://oval.mitre.org/XMLSchema/oval-definitions-5)

Unknown XML namespaces are ignored. Any attempt to load a benchmark with an unknown DataStream or XCCDF version will thus yield a missing XCCDF error.

Useful links

Glossary

SCAP

The Security Content Automation Protocol (SCAP) standard includes several component standards designed to work together. For each component, the standard defines a document format with the syntax and semantics of the internal data structures. All component standards are based on XML, and each component standard defines its own XML namespace.

XCCDF

XCCDF (Extensible Configuration Checklist Description Format) is the language used to describe security checklists. The language is designed to support information interchange, document generation, organizational and situational tailoring, automated compliance testing, and compliance scoring.

The language does not contain any commands to perform the analysis and it is mostly descriptive. Other component documents (OVAL and SCE scripts) can be referenced from the XCCDF document.

OVAL

OVAL (Open Vulnerability and Assessment Language) is a declarative language for making logical assertions about the state of the system. It is used to describe security vulnerabilities or the desired configuration of systems. OVAL definitions define a secure state of some objects in a computer, for example configuration files, file permissions, processes. OVAL definitions are evaluated using an interpreter called scanner.

The main concepts of the OVAL document are definitions, tests, objects and states. Each definition specifies the tests that need to be satisfied. A test links an object and a state. Test will pass when a resource denoted by given object satisfies requirements in a corresponding state.

SCE

SCE (Script Check Engine) is an extension allowing script execution (written for example in Bash, PowerShell, …) from the SCAP policy. It allows you to make your scripts interoperable with your security policy.

DataStream

DataStream is a format for packing into a single XML file the different SCAP components of the benchmark, in our case: XCCDF and OVAL.

CPE

The CPE (Common Platform Enumeration) is used to identify platforms and computer systems using unequivocally defined names.