Benchmarks

A compliance analysis compares the target system to a reference (the benchmark) which is a set of security rules to assess the compliance or non-compliance of the system.

A benchmark is defined using the SCAP standard that allows organizations to automate how they monitor vulnerabilities and ensure they are compliant with security policies.

Structure of a benchmark

A benchmark includes an XCCDF file that describes the compliance rules and the tests to evaluate them. These tests can be written in OVAL or SCE format.

Cyberwatch supports two benchmark formats:

  • a ZIP file containing at least an XCCDF file and possibly other files (SCE or OVAL scripts) whose relative paths have been declared in the XCCDF,
  • a file in DataStream format.

Minimalist example of XCCDF file

Example XCCDF file extracted from the CIS benchmark for Ubuntu Linux 22.04 LTS that defines a benchmark containing only one compliance rule:

<xccdf:Benchmark id="xccdf_org.cisecurity.benchmarks_benchmark_1.0.0_CIS_Ubuntu_Linux_22.04_LTS_Benchmark" xmlns:xccdf="http://checklists.nist.gov/xccdf/1.2" xmlns="http://checklists.nist.gov/xccdf/1.2" >
  <xccdf:title>CIS Ubuntu Linux 22.04 LTS Benchmark</xccdf:title>
  <xccdf:platform idref="cpe:2.3:o:canonical:ubuntu_linux:22.04:*:*:*:lts:*:*:*"/>
  <xccdf:Rule id="xccdf_org.cisecurity.benchmarks_rule_1.1.5.3_Ensure_noexec_option_set_on_varlog_partition" role="full" selected="false" weight="1.0">
    <xccdf:title xml:lang="en">Ensure noexec option set on /var/log partition</xccdf:title>
    <xccdf:description xml:lang="en">
      <xhtml:p><xhtml:span class="inline_block">noexec</xhtml:span> mount option specifies that the filesystem cannot contain executable binaries.</xhtml:p>
    </xccdf:description>
    <xccdf:rationale xml:lang="en">
      <xhtml:p> Since the <xhtml:span class="inline_block">/var/log</xhtml:span> filesystem is only intended for log files, set this option to ensure that users cannot run executable binaries from <xhtml:span class="inline_block">/var/log</xhtml:span>.</xhtml:p>
    </xccdf:rationale>
    <xccdf:fixtext xml:lang="en">
      <xhtml:div>
        <xhtml:p><xhtml:strong>IF</xhtml:strong> the <xhtml:span class="inline_block">/var/log</xhtml:span> partition exists, edit the <xhtml:span class="inline_block">/etc/fstab</xhtml:span> file and add <xhtml:span class="inline_block">noexec</xhtml:span> to the fourth field (mounting options) for the <xhtml:span class="inline_block">/var/log</xhtml:span> partition.</xhtml:p>
        <xhtml:p>Run the following command to remount <xhtml:span class="inline_block">/var/log</xhtml:span> with the configured options:    </xhtml:p>
        <xhtml:code class="code_block"># mount -o remount /var/log</xhtml:code>
      </xhtml:div>
    </xccdf:fixtext>
    <xccdf:complex-check operator="AND">
      <check system="http://oval.mitre.org/XMLSchema/oval-definitions-5">
        <check-content-ref href="CIS_Ubuntu_Linux_22.04_LTS_Benchmark_v1.0.0-oval.xml" name="oval:org.cisecurity.benchmarks.canonical_ubuntu_linux_22:def:2956581"/>
      </check>
    </xccdf:complex-check>
  </xccdf:Rule>
</xccdf:Benchmark>

A few elements to note in this document:

  • The benchmark identifier defined by the id attribute on the <xccdf:Benchmark> tag which serves as a reference for Cyberwatch.
  • The target platform is defined in CPE format by the idref attribute on the <xccdf:platform> tag. It is also possible to declare one or more platforms at the rule level if you want to restrict its application.
  • In the definition of the rule <xccdf:Rule>, the content of the tag <xccdf:complex-check> is used to list the tests to run and the relative paths to access the files where they are defined.

In this example, the only rule references a test in OVAL format, the benchmark will therefore contain this XCCDF file and the declared OVAL file. It can then be distributed in ZIP format or in DataStream format.

On the other hand, if the rule had referred to a script in SCE format, by declaring for example:

<check system="http://open-scap.org/page/SCE">
  <check-import import-name="stdout"/>
  <check-content-ref href="sce/ensure_no_exec_option.sh"/>
</check>

The benchmark would contain the XCCDF file and a sce directory containing the Shell script ensure_no_exec_option.sh. All files should be compressed into a ZIP file in order to be added to Cyberwatch, as described below.

The tree structure of the files in the ZIP file must imperatively respect the relative paths declared in the XCCDF file.

Add a benchmark

  1. Go to Settings > Benchmarks
  2. Click « Add »
  3. Select the benchmark to add, in ZIP or DataStream format
  4. Click « Import »

During import, if the benchmark identifier already exists in Cyberwatch, the benchmark is updated. Otherwise, a new benchmark is created.

It is possible to import new benchmarks only for the platforms supported by the compliance management module of Cyberwatch. It is therefore possible to import a benchmark for other versions of operating systems such as macOS or Ubuntu 12.04 for example, but not to import a benchmark to check a software configuration or network equipment for example.

List of supported platforms

The following table lists the supported platforms and the benchmarks already installed:

Operating systemCPE of the platformInstalled benchmarks
AlmaLinuxcpe:/o:almalinux:almalinux:8, …CIS Alma Linux OS 8 Benchmark
CIS AlmaLinux OS 9 Benchmark
Amazon Linux 2cpe:/o:amazon:linux_2CIS Amazon Linux 2 Benchmark
Apple macOScpe:/o:apple:mac_os_x:10.15.0, …CIS Apple macOS 10.15 Catalina Benchmark
CIS Apple macOS 11.0 Big Sur Benchmark
CIS Apple macOS 12.0 Monterey Benchmark
CIS Apple macOS 13.0 Ventura Benchmark
CentOS Linuxcpe:/o:centos:centos:6, …CIS CentOS Linux 6 Benchmark
CIS CentOS Linux 7 Benchmark
Debiancpe:/o:debian:debian_linux:9.0, …CIS Debian Linux 9 Benchmark
CIS Debian Linux 10 Benchmark
CIS Debian Linux 11 Benchmark
Fedora Linuxcpe:/o:fedora:linux:28CIS Fedora 28 Family Linux Benchmark
Microsoft Windows 8cpe:/o:microsoft:windows_8CIS Microsoft Windows 8 Benchmark
Microsoft Windows 10cpe:/o:microsoft:windows_10CIS Microsoft Windows 10 Enterprise Benchmark
CIS Microsoft Intune for Windows 10 Benchmark
Microsoft Windows 11cpe:/o:microsoft:windows_11CIS Microsoft Windows 11 Enterprise Benchmark
CIS Microsoft Intune for Windows 11 Benchmark
Microsoft Windows Server 2008cpe:/o:microsoft:windows_server_2008,
cpe:/o:microsoft:windows_server_2008:r2
CIS Microsoft Windows Server 2008 (non-R2) Benchmark
CIS Microsoft Windows Server 2008 R2 Benchmark
Microsoft Windows Server 2012cpe:/o:microsoft:windows_server_2012,
cpe:/o:microsoft:windows_server_2012:r2
CIS Microsoft Windows Server 2012 (non-R2) Benchmark
CIS Microsoft Windows Server 2012 R2 Benchmark
Microsoft Windows Server 2016cpe:/o:microsoft:windows_server_2016CIS Microsoft Windows Server 2016 Benchmark
Microsoft Windows Server 2019cpe:/o:microsoft:windows_server_2019CIS Microsoft Windows Server 2019 Benchmark
Microsoft Windows Server 2022cpe:/o:microsoft:windows_server:2022CIS Microsoft Windows Server 2022 Benchmark
Oracle Linuxcpe:/o:oracle:linux:6, …CIS Oracle Linux 6 Benchmark
CIS Oracle Linux 7 Benchmark
CIS Oracle Linux 8 Benchmark
CIS Oracle Linux 9 Benchmark
Red Hat Enterprise Linuxcpe:/o:redhat:redhat_enterprise_linux:6, …CIS Red Hat Enterprise Linux 6 Benchmark
CIS Red Hat Enterprise Linux 7 Benchmark
CIS Red Hat Enterprise Linux 8 Benchmark
CIS Red Hat Enterprise Linux 9 Benchmark
Rocky Linuxcpe:/o:rocky:rocky_linux:8, …CIS Rocky Linux 8 Benchmark
CIS Rocky Linux 9 Benchmark
SUSE Linux Enterprise Servercpe:/o:suse:suse_linux_enterprise_server:12, …CIS SUSE Linux Enterprise 12 Benchmark
CIS SUSE Linux Enterprise 15 Benchmark
Ubuntucpe:/o:canonical:ubuntu_linux:16.04, …CIS Ubuntu Linux 14.04 LTS Benchmark
CIS Ubuntu Linux 16.04 LTS Benchmark
CIS Ubuntu Linux 18.04 LTS Benchmark
CIS Ubuntu Linux 20.04 LTS Benchmark
CIS Ubuntu Linux 22.04 LTS Benchmark
Linux kernelcpe:/o:linux:linux_kernel:*CIS Docker Benchmark

Supported standards

The following SCAP components, with their XML namespace, are supported:

  • DataStream 1.2 (http://scap.nist.gov/schema/scap/source/1.2),
  • XCCDF 1.2 (http://checklists.nist.gov/xccdf/1.2),
  • OVAL 5 (http://oval.mitre.org/XMLSchema/oval-definitions-5).

Unknown XML namespaces are ignored. Any attempt to load a benchmark with an unknown DataStream or XCCDF version will thus yield a missing XCCDF error.

Glossary

SCAP

The Security Content Automation Protocol (SCAP) standard includes several component standards designed to work together. For each component, the standard defines a document format with the syntax and semantics of the internal data structures. All component standards are based on XML, and each component standard defines its own XML namespace.

XCCDF

XCCDF (Extensible Configuration Checklist Description Format) is the language used to describe security checklists. The language is designed to support information interchange, document generation, organizational and situational tailoring, automated compliance testing, and compliance scoring.

The language does not contain any commands to perform the analysis and it is mostly descriptive. Other component documents (OVAL and SCE scripts) can be referenced from the XCCDF document.

OVAL

OVAL (Open Vulnerability and Assessment Language) is a declarative language for making logical assertions about the state of the system. It is used to describe security vulnerabilities or the desired configuration of systems. OVAL definitions define a secure state of some objects in a computer, for example configuration files, file permissions, processes. OVAL definitions are evaluated using an interpreter called scanner.

The main concepts of the OVAL document are definitions, tests, objects and states. Each definition specifies the tests that need to be satisfied. A test links an object and a state. Test will pass when a resource denoted by given object satisfies requirements in a corresponding state.

SCE

SCE (Script Check Engine) is an extension allowing script execution (written for example in Bash, PowerShell, …) from the SCAP policy. It allows you to make your scripts interoperable with your security policy.

DataStream

DataStream is a format for packing into a single XML file the different SCAP components of the benchmark, in our case: XCCDF and OVAL.

CPE

The CPE (Common Platform Enumeration) is used to identify platforms and computer systems using unequivocally defined names.