Skip to main content Link Menu Expand (external link) Document Search Copy Copied

Benchmarks

A compliance analysis compares the target system to a reference (the benchmark) which is a set of security rules to assess the compliance or non-compliance of the system.

A benchmark is defined using the SCAP standard that allows organizations to automate how they monitor vulnerabilities and ensure they are compliant with security policies.

Structure of a benchmark

A benchmark includes an XCCDF file that describes the compliance rules and the tests to evaluate them. These tests can be written in OVAL or SCE format.

Cyberwatch supports two benchmark formats:

  • a ZIP file containing at least an XCCDF file and possibly other files (SCE or OVAL scripts) whose relative paths have been declared in the XCCDF,
  • a file in DataStream format.

Minimalist example of XCCDF file

Example XCCDF file extracted from the CIS benchmark for Ubuntu Linux 22.04 LTS that defines a benchmark containing only one compliance rule:

<xccdf:Benchmark id="xccdf_org.cisecurity.benchmarks_benchmark_1.0.0_CIS_Ubuntu_Linux_22.04_LTS_Benchmark" xmlns:xccdf="http://checklists.nist.gov/xccdf/1.2" xmlns="http://checklists.nist.gov/xccdf/1.2" >
  <xccdf:title>CIS Ubuntu Linux 22.04 LTS Benchmark</xccdf:title>
  <xccdf:platform idref="cpe:2.3:o:canonical:ubuntu_linux:22.04:*:*:*:lts:*:*:*"/>
  <xccdf:Rule id="xccdf_org.cisecurity.benchmarks_rule_1.1.5.3_Ensure_noexec_option_set_on_varlog_partition" role="full" selected="false" weight="1.0">
    <xccdf:title xml:lang="en">Ensure noexec option set on /var/log partition</xccdf:title>
    <xccdf:description xml:lang="en">
      <xhtml:p><xhtml:span class="inline_block">noexec</xhtml:span> mount option specifies that the filesystem cannot contain executable binaries.</xhtml:p>
    </xccdf:description>
    <xccdf:rationale xml:lang="en">
      <xhtml:p> Since the <xhtml:span class="inline_block">/var/log</xhtml:span> filesystem is only intended for log files, set this option to ensure that users cannot run executable binaries from <xhtml:span class="inline_block">/var/log</xhtml:span>.</xhtml:p>
    </xccdf:rationale>
    <xccdf:fixtext xml:lang="en">
      <xhtml:div>
        <xhtml:p><xhtml:strong>IF</xhtml:strong> the <xhtml:span class="inline_block">/var/log</xhtml:span> partition exists, edit the <xhtml:span class="inline_block">/etc/fstab</xhtml:span> file and add <xhtml:span class="inline_block">noexec</xhtml:span> to the fourth field (mounting options) for the <xhtml:span class="inline_block">/var/log</xhtml:span> partition.</xhtml:p>
        <xhtml:p>Run the following command to remount <xhtml:span class="inline_block">/var/log</xhtml:span> with the configured options:    </xhtml:p>
        <xhtml:code class="code_block"># mount -o remount /var/log</xhtml:code>
      </xhtml:div>
    </xccdf:fixtext>
    <xccdf:complex-check operator="AND">
      <check system="http://oval.mitre.org/XMLSchema/oval-definitions-5">
        <check-content-ref href="CIS_Ubuntu_Linux_22.04_LTS_Benchmark_v1.0.0-oval.xml" name="oval:org.cisecurity.benchmarks.canonical_ubuntu_linux_22:def:2956581"/>
      </check>
    </xccdf:complex-check>
  </xccdf:Rule>
</xccdf:Benchmark>

A few elements to note in this document:

  • The benchmark identifier defined by the id attribute on the <xccdf:Benchmark> tag which serves as a reference for Cyberwatch.
  • The target platform is defined in CPE format by the idref attribute on the <xccdf:platform> tag.
  • In the definition of the rule <xccdf:Rule>, the content of the tag <xccdf:complex-check> is used to list the tests to run and the relative paths to access the files where they are defined.

In this example, the only rule references a test in OVAL format, the benchmark will therefore contain this XCCDF file and the declared OVAL file. It can then be distributed in ZIP format or in DataStream format.

On the other hand, if the rule had referred to a script in SCE format, by declaring for example:

<check system="http://open-scap.org/page/SCE">
  <check-import import-name="stdout"/>
  <check-content-ref href="sce/ensure_no_exec_option.sh"/>
</check>

The benchmark would contain the XCCDF file and a sce directory containing the Shell script ensure_no_exec_option.sh. All files should be compressed into a ZIP file in order to be added to Cyberwatch, as described below.

The tree structure of the files in the ZIP file must imperatively respect the relative paths declared in the XCCDF file.

Add a benchmark

  1. Go to Settings > Benchmarks
  2. Click « Add »
  3. Select the benchmark to add, in ZIP or DataStream format
  4. Click « Import »

During import, if the benchmark identifier already exists in Cyberwatch, the benchmark is updated. Otherwise, a new benchmark is created.

N.B. It is possible to import new benchmarks only for the platforms supported by the compliance management module of Cyberwatch. It is therefore possible to import a benchmark for other versions of operating systems such as macOS or Ubuntu 12.04 for example, but not to import a benchmark to check a software configuration or network equipment for example.

List of supported platforms

The following table lists the supported platforms and the benchmarks already installed:

Operating system CPE of the platform Installed benchmarks
AlmaLinux cpe:/o:almalinux:almalinux:8 CIS Alma Linux OS 8 Benchmark
Amazon Linux 2 cpe:/o:amazon:linux_2 CIS Amazon Linux 2 Benchmark
Apple macOS cpe:/o:apple:mac_os_x:12.0.0 CIS Apple macOS 12.0 Monterey Benchmark
CentOS Linux cpe:/o:centos:centos:6, … CIS CentOS Linux 6 Benchmark
CIS CentOS Linux 7 Benchmark
Debian cpe:/o:debian:debian_linux:9.0, … CIS Debian Linux 9 Benchmark
CIS Debian Linux 10 Benchmark
CIS Debian Linux 11 Benchmark
Fedora Linux cpe:/o:fedora:linux:28 CIS Fedora 28 Family Linux Benchmark
Microsoft Windows 8 cpe:/o:microsoft:windows_8 CIS Microsoft Windows 8 Benchmark
Microsoft Windows 10 cpe:/o:microsoft:windows_10 CIS Microsoft Windows 10 Enterprise Benchmark
Microsoft Windows 11 cpe:/o:microsoft:windows_11 CIS Microsoft Windows 11 Enterprise Benchmark
Microsoft Windows Server 2008 cpe:/o:microsoft:windows_server_2008,
cpe:/o:microsoft:windows_server_2008:r2
CIS Microsoft Windows Server 2008 (non-R2) Benchmark
CIS Microsoft Windows Server 2008 R2 Benchmark
Microsoft Windows Server 2012 cpe:/o:microsoft:windows_server_2012,
cpe:/o:microsoft:windows_server_2012:r2
CIS Microsoft Windows Server 2012 (non-R2) Benchmark
CIS Microsoft Windows Server 2012 R2 Benchmark
Microsoft Windows Server 2016 cpe:/o:microsoft:windows_server_2016 CIS Microsoft Windows Server 2016 Benchmark
Microsoft Windows Server 2019 cpe:/o:microsoft:windows_server_2019 CIS Microsoft Windows Server 2019 Benchmark
Microsoft Windows Server 2022 cpe:/o:microsoft:windows_server:2022 CIS Microsoft Windows Server 2022 Benchmark
Oracle Linux cpe:/o:oracle:linux:6, … CIS Oracle Linux 6 Benchmark
CIS Oracle Linux 7 Benchmark
CIS Oracle Linux 8 Benchmark
Red Hat Enterprise Linux cpe:/o:redhat:redhat_enterprise_linux:6, … CIS Red Hat Enterprise Linux 6 Benchmark
CIS Red Hat Enterprise Linux 7 Benchmark
CIS Red Hat Enterprise Linux 8 Benchmark
Rocky Linux cpe:/o:rocky:rocky_linux:8 CIS Rocky Linux 8 Benchmark
SUSE Linux Enterprise Server cpe:/o:suse:suse_linux_enterprise_server:12, … CIS SUSE Linux Enterprise 12 Benchmark
CIS SUSE Linux Enterprise 15 Benchmark
Ubuntu cpe:/o:canonical:ubuntu_linux:16.04, … CIS Ubuntu Linux 14.04 LTS Benchmark
CIS Ubuntu Linux 16.04 LTS Benchmark
CIS Ubuntu Linux 18.04 LTS Benchmark
CIS Ubuntu Linux 20.04 LTS Benchmark
CIS Ubuntu Linux 22.04 LTS Benchmark

Supported standards

The following SCAP components, with their XML namespace, are supported:

  • DataStream 1.2 (http://scap.nist.gov/schema/scap/source/1.2),
  • XCCDF 1.2 (http://checklists.nist.gov/xccdf/1.2),
  • OVAL 5 (http://oval.mitre.org/XMLSchema/oval-definitions-5).

Unknown XML namespaces are ignored. Any attempt to load a benchmark with an unknown DataStream or XCCDF version will thus yield a missing XCCDF error.

Glossary

SCAP

The Security Content Automation Protocol (SCAP) standard includes several component standards designed to work together. For each component, the standard defines a document format with the syntax and semantics of the internal data structures. All component standards are based on XML, and each component standard defines its own XML namespace.

XCCDF

XCCDF (Extensible Configuration Checklist Description Format) is the language used to describe security checklists. The language is designed to support information interchange, document generation, organizational and situational tailoring, automated compliance testing, and compliance scoring.

The language does not contain any commands to perform the analysis and it is mostly descriptive. Other component documents (OVAL and SCE scripts) can be referenced from the XCCDF document.

OVAL

OVAL (Open Vulnerability and Assessment Language) is a declarative language for making logical assertions about the state of the system. It is used to describe security vulnerabilities or the desired configuration of systems. OVAL definitions define a secure state of some objects in a computer, for example configuration files, file permissions, processes. OVAL definitions are evaluated using an interpreter called scanner.

The main concepts of the OVAL document are definitions, tests, objects and states. Each definition specifies the tests that need to be satisfied. A test links an object and a state. Test will pass when a resource denoted by given object satisfies requirements in a corresponding state.

SCE

SCE (Script Check Engine) is an extension allowing script execution (written for example in Bash, PowerShell, …) from the SCAP policy. It allows you to make your scripts interoperable with your security policy.

DataStream

DataStream is a format for packing into a single XML file the different SCAP components of the benchmark, in our case: XCCDF and OVAL.

CPE

The CPE (Common Platform Enumeration) is used to identify platforms and computer systems using unequivocally defined names.