Audit Results Session 2014.1

Audit Type	Compliance Review
Report Status	Final Report
Audit initiated by	Audit Plan
Audit Subject	Arbitrated Background Check
Follow up status	2015-08-26 informed board, management review expected till 2015-09-30
	2015-12-06 approved by board in m20151206.8

Contents

Audit Results Session 2014.1

Executive Summary

CAcert conducts Arbitrated Background Checks since 2010 to only grant vetted people access to sensitive areas such as root keys, server systems and mass data handling. Roles explicit mentioned are: Critical Systems Administration, Support Personnel, Access Engineers and Software Assessors.

Five recommendations have been identified.

Purpose, Scope and Methodology

The Audit of the methods uses performing an arbitrated background check (ABC) is part of the current Audit Programme 2014 - 2016.

Scope of this session is to test the ABC against binding policies and to determine the use of the ABC to the desired outcome, to enhance the security of CAcert when installing personnel in critical positions such as (critical) infrastructure administrators or software assessors.

To conduct the audit, an inspection of an ABC interview (related to a20140124.1) and the related policies and documents was done.

The following documents have been taken into account:

Security Policy (SP) Art. 9.1.4
Background Check Procedure (BCP)
http://wiki.cacert.org/SecurityManual#Background_Check_Procedures (SM)
Philipp Dunkel's thoughts on Arbitrated Background Checks
ABC questionnaire (requested)

Audit Results and Recommendations

Requirements for an ABC are listed in the table below:

No.	Source	Requirement	Fulfilment in a20140124.1
R1	SP 9.1.4	1 Arbitrator, 1 Case Manager (not in personal union)	{+}
R2	SP 9.1.4.1	Candidate needs realm-specific knowledge	{-}
R3	SP 9.1.4.1	Candidate needs realm-specific understanding of good security practice	{-}
R4	SP 9.1.4.1	Candidate needs a history of activity within Community	{0}
R5	SP 9.1.4.1	Candidate needs reputation and standing within Community	{0}
R6	SP 9.1.4.1	Candidate provided references needs to be verified	{0}
R7	SP 9.1.4.1	Candidate must be asked and checked for conflicts of interests	{+}
R8	SP 9.1.4.3	Process should be documented as a procedure	{+} [1]
R9	SP 9.1.4.3	Documentation of individual check should be preserved	{+}
R10	SP 9.1.4.4	Procedure and ruling (recommendation) should be public	{+}
R11	SP 9.1.4.4	Interview, documents should not be public	{+}
R12	SP 9.1.4.4	Summary of evidence should be in the ruling	{+}
R13	SP 9.1.4.4	Arbitrator can rule on the escrow questions of evidence	{+}
R14	SP 9.1.4.4	Contact information goes into the contact addressbook	N/A
R15	BCP 1.1	Independence of Arbitrator from candidate	{+}
R16	BCP 1.2	A dispute against the candidate must exist	{+}
R17	BCP 2.2	Candidate must provide a curriculum vitae	{+}
R18	BCP 2.4	Candidate must stand an interview (, telephone call, or chat)	{+}
R19	BCP 3	Ruling must contain reliability of the candidate	N/A
R20	BCP 3	Ruling must contain further needed training in data security, social engineering or other fields	N/A

Note: The fulfilment does not say, if the candidate fulfils any of the requirements, but the ABC itself covered this topics

[1] Refers to Background Check Procedure

While the {+} marked requirements are handled in compliance, only requirements marked with {0} and {-} are addressed. Requirements with {-} are not compliant and can lead to a non-conformity, while {0} marked requirements are not evaluable by the methods used during audit and might lead to a recommendation.

Requirements R4 and R5 are evident for the candidate of this ABC, since the candidate is known to the Case Manager, the Arbitrator and the Auditor before. Nonetheless, the auditor was not able to validate the reputation check during the course of the audit.
The Arbitrator stated, he validated the candidate's curriculum vitae (R6) with the help of a internet search engine. He especially checked the academic education of the candidate. The Auditor is unsure, if this is enough for an ABC.
The candidate's realm-specific knowledge (R2) was not checked during the ABC interview, especially to software assessment / development capabilities.
The candidate's references (R3) have not been checked in the proceeding of the ABC.

The ABC over the candidate was for the purpose of working as an Arbitrator. However, there were no "realm-specific" questions to the candidate, even as the SP clearly asks for them (see R2 & R3). Specific questions about the candidate’s life, especially where she is assailable, have not been covered.

Furthermore, the questionnaire used for the ABC was less specific for the prospective role of the candidate, nor hard to answer. Some of the questions had nothing to do with CAcert or the subjects of protection. This questionnaire needs a rework to be applicable for ABCs again.

Non-Conformities

none

Recommendations

For the SP, an ABC is always with a purpose: Critical Systems Administration, Support Personnel, Access Engineers or Software Assessors. An ABC with no or a purpose not named in SP should not exist.
The Arbitrator should always check and document all steps taken to validate R2 - R7.
The Arbitrator should use all available resources (search engines, publications, professional social networks, references, employees) to validate and verify a candidate's history if relevant for the special case. Especially, to proof abilities in a special field.
The Arbitrator should put the more attention to the candidate’s strengthens and weaknesses and ask specific questions there to determine the risk for CAcert.
Complete rework of the ABC questionnaire, specific questions for the roles Software Assessor, Access Engineer, (Critical) System Administrator, and Support Personnel should be attached. The questionnaire should be checked regularly on applicability to reflect changes, threads and risk within and outside of CAcert.

-- BenediktHeintel 2015-08-26 21:36:40

Audit/Results/session2014.1 (last edited 2015-12-08 23:06:46 by BenediktHeintel)