-g option removed

[signduterre.git] / signduterre.py

#!/usr/bin/python
manual = """
Signature-du-Terroir
Construct a signature of the installed software state or check the integrity of the installation
using a previously made signature.

Usage: signduterre.py [options] FILE1 FILE2 ...

Options:
  -h, --help            show this help message and exit
  -s HEX, --salt=HEX    Enter salt in cleartext. If not given, a hexadecimal
                        salt will be suggested. The SUGGESTED[=N] keyword will
                        cause the selection of the suggested string. N is the
                        number of salts generated (default N=1)
  -p TEXT, --passphrase=TEXT
                        Enter passphrase in cleartext. If not given, a cleartext
                        passphrase will be suggested. The keyword SUGGESTED
                        will cause the suggested passphrase to be used.
                        Entering the name of an existing file will cause it to
                        be read and a random passphrase found in the file will 
                                                be used (creating a signature), or they will all be used 
                                                in sequence (--check-file).
  -c FILE, --check-file=FILE
                        Check contents of file (output of previous run)
  -i FILE, --input-file=FILE
                        Use names from input-file (one filename per line)
  -u USER, --user=USER  Execute $(cmd) as USER, default 'nobody' (root/sudo
                        only)
  -S, --Status          For each file, add a line with unvarying file status
                        information: st_mode, st_ino, st_dev, st_uid, st_gid,
                        and st_size (like the '?' prefix, default False)
  -t, --total-only      Only print the total hash, must be checked BEFORE
                        running --detail (default True)
  -d, --detailed-view   Print hashes of individual files, must be checked
                        AFTER running --total-only check (default False)
  -e, --execute         Interpret $(cmd) (default False)
  -n, --no-execute      Explicitely do NOT Interpret ${ENV} and $(cmd)
  -m, --manual          Print the manual and exit
  -r, --release-notes   Print the release notes and exit
  -l, --license         Print license text and exit
  -v, --verbose         Print more information
  
FILE1 FILE2 ...
Names and paths of one or more files to be checked. Any name starting with a '$', eg, $PATH, will be
interpreted as an environmental variable or command according to the bash conventions:
'$ENV' and '${ENV}' as variables, '$(cmd;cmd...)' as system commands (bash --restricted -c 'cmd;cmd...' PID).
Where PID the current Process ID is (available as positional parameter $0). Do not forget to enclose the 
arguments in single ''-quotes! The commands are scanned for unwanted characters (eg, ') and these are removed. 
The use of '$(cmd;cmd...)' requires explicit use of the -e or --execute option.
 
If executed as root or sudo, $(cmd;cmd...) will be executed as 'sudo -H -u <user>' which defaults to
--user nobody ('--user root' is at your own risk). This will obviously not work when invoked as non-root/sudo.
--user root is necessary when you need to check privileged information, eg, 
you want to check the MBR with '$(dd if=/dev/hda bs=512 count=1 | od -X)'
However, as you might use --check-file with files you did not create yourself, it is important to
be warned if commands are to be executed.

Interpretation of $() ONLY works if the -e or --execute options are entered. signduterre.py can easily 
be adapted to automatically use the setting in the check-file. However, this is deemed insecure and 
commented out in the distribution version.

The -n or --no-execute option explicitely supress the interpretation of $(cmd) arguments.

Meta information from lstat on files is signed when the filename is preceded by a '?'. '?./signduterre.py' will 
extract (st_mode, st_ino, st_dev, st_nlinks, st_uid, st_gid, st_size) and hash a line of these data (visible with --verbose).
The --Status option will automatically add such a line in front of every file. Note that '?' is implied for directories.
both '/' and '?/' produce a hash of, eg,: 
lstat(/) = [st_mode=041775, st_ino=2, st_dev=234881026, st_uid=0, st_gid=80, st_size=1360]
Note that nlinks of a directory include every file in the directory, so this will check whether files have been added
to a directory.

Signature-du-Terroir

A very simple security application to test for the integrity of files and "states" in a computer installation.
signduterre.py constructs a signature of the current system state and checks installation state with a previously made signature.
The files are hashed with a passphrase to allow detection of compromised systems while running on the same system.
The signature checking can be subverted, but the flexibillity of signduterre.py and the fact that the output of any command
can be tested should hamper automated root-kit attacks.

signduterre.py writes a total SHA-256 hash to STDOUT of all the files and commands entered as arguments. It can also write a 
hash for each individual file (insecure). The output of a signature can be send to a file and later used to 
check with --check-file. Hashes are calculated with a hashed salt + passphrase sequence pre-pended to create 
unpredictable hashes. This procedure ensures that an attacker does not know whether or not the correct passphrase 
has been entered. An attacker can only know when to supply the requested hash values if she knows
the passphrase or has copies available of all the tested files and output of commands to calculate the hashes 
on the fly.

SECURITY WARNINGS:

When run on a compromised system, signduterre.py can be subverted if the attacker keeps a copy of all the files and 
reroutes the open() and lstat() functions, or simply delegating signduterre.py to a chroot jail with the original system.
In principle, signduterre.py only checks whether the computer responds identically to when the sinature file was made.
There is no theoretic barrier against a compromised computer perfectly simulating the original system when tested,
but behaving adversely at other times. Except for running from clean boot media (USB?), I know of no theoretical 
sound solution to this problem ;-) 

However, this scenario assumes the use of unlimited resources and time. Inside a limited, real computer system, the 
attacker must make compromises on what can and what cannot be simulated with the available time and hardware. The idea 
behind signduterre.py is to "ask difficult questions" that increase the cost of simulating the original system 
high enough to make detection of successful attacks likely. But signduterre.py simply intended to raise the bar. 
One point is to store the times needed to create the original hashes. This timing can later be used to see whether 
the new timings are reasonable. If the same hardware takes considerably longer to perform the same calculations,
or needs a much longer delay before it starts, the tester might want to see where this time is spent.

Signature-du-Terroir works on the assumption that any attacker in control of a compromised system cannot
predict whether the passphrase entered is correct or not. An attacker can always intercept the in- and output of 
signduterre. When running a --check-file, this means the program can be made to print out OK irrespective of the
tests. A safe use of signduterre.py is to start with a random number of incorrect passphrases and see whether they fail. 
Repeat: 
THE CORRECT USE OF signduterre.py IS TO ENTER A RANDOM NUMBER OF INCORRECT PASSPHRASES FOR EACH TEST
AND SEE WHETHER IT FAILS EVERY TIME!

On a compromised system, signduterre.py's detailed file testing (--detailed-view) is easily subverted. With a 
matched file hash, the attacker will know that the correct passphrase has been entered and can print out the 
stored hashes or 'ok's  for the rest of the checks. So if the attacker keeps any entry in the signature file 
uncompromised, she can intercept the output, test the password on the unchanged entry and substitute the 
requested hashes for the output if the hash of that entry matches. 

When checking for root-kits and other malware, it is safest to compare the signature files from a different, 
clean, system. But then you would not need signduterre.py anyway. 
If you have to work on the system itself, only use the -t or --total-only options to create 
signatures with a total hash and without individual file hashes. Such a signature can be used to check 
whether the system is unchanged. Another signature file WITH A DIFFERENT PASSPHRASE can then be used to 
identify the individual files that have changed. If a detailed signature file has the same passphrase,
an attacker could use that other file to read the individual file hashes to check whether the correct 
passphrase was entered.

Using the --check-file option in itself is perfectly UNsafe. An attacker simply has to print out 'OK' and
to defeat the check. This attack can be foiled by making it unpredictable when signduterre.py should return
'OK'. This can be done by using a list of salts or passphrases where only one of them (or none!) is correct.
Any attacker will have to guess when to return 'OK'.

As generating and entering wrong passphrases and salts is tedious, it is to be expected that users will 
take shortcuts. To assist users, the '--salt SUGGESTED=<N>' option will generate a number N of salts. When 
checking, each of these salts is tried in turn. An attacker that is unable to simulate the uncompromised 
system will have to guess which one of the salts is the correct one, and whether or not the passphrase 
is correct. This increases the chances of detecting compromised systems.

The '--passphrase SUGGESTED=N' option will generate and print N passphrases. One of these is chosen at 
random for the signature. The number of the chosen passphrase is printed on STDERR with the passwords. 
When checking a file, the stored passphrases can be read in again, either by entering the passphrase 
file after the --passphrase option ('--passphrase <passphrase file>'), or directly from the --check-file. 
signduterre.py will print out the result for each of the passphrases. 

Note, that storing passphrases in a file and feeding it to signduterre.py is MUCH less secure than just 
typing them in. Moreover, it might completely defeat the purpose of signduterre.py. If future experiences 
cast any more doubt on the security ofthis option, it will be removed. 

For those who want to know more about what an "ideal attacker" can do, see:
Ken Thompson "Reflections on Trusting Trust" 
http://www.acm.org/classics/sep95/

David A Wheeler "Countering Trusting Trust through Diverse Double-Compiling"
http://www.acsa-admin.org/2005/abstracts/47.html

and the discussion of these at Bruce Schneier's 'Countering "Trusting Trust"'
http://www.schneier.com/blog/archives/2006/01/countering_trus.html

Manual

The intent of signduterre.py is to ensure that the signature cannot be subverted even if the system has been compromised
by an attacker that has obtained root control over the computer and any existing signature files.

signduterre.py asks for a passphrase which is PRE-pended to every file before the hash is constructed (unless the
passphrase is entered with an option). As long as the passphrase is not compromised, the hashes cannot 
be reconstructed.  A randomly generated, unpadded base-64 encoded 16 Byte password (ie, ~22 characters) is suggested in 
interactive use. If '--passphrase SUGGESTED' is entered on the command line as the salt, the suggested 
value will be used. This value is printed to STDERR (the screen or 2) for safe keeping. Please, make sure
you store the printed passphrase. For instance:
python signduterre.py -p SUGGESTED -s SUGGESTED /boot/* /sbin/* /bin/* \\
2> Signature_`date "+%Y%m%d_%H-%M-%S"`.pwd > Signature_`date "+%Y%m%d_%H-%M-%S"`.txt
will store the passphrase (and all error messages) in a file like 'Signature_20090630_11-14-03.pwd'
and the checik-file in 'Signature_20090630_11-14-03.txt'. 

It is not secure to store files with the passphrase on the system you want to check. However, you could
pipe STDERR to some safe site.

Good passphrases are difficult to remember, so their plaintext form should be protected. To protect the 
passphrase against rainbow and brute force attacks, the passphrase is concatenated to a salt phrase and 
hashed before use (SHA-256). 

The salt phrase is requested when constructing a signature. In interactive use, an 8 byte hexadecimal 
(= 16 character) salt from /dev/random is suggested. If '--salt SUGGESTED' is entered on the command line
as the salt, the suggested value will be used. The salt is printed in plaintext to the output. 
The salt will make it more difficult to determine whether the same passphrase has been used to create
different signatures.

At the bottom, a 'TOTAL HASH' line will be printed that hashes all the lines printed for the files. This includes
the file names as printed on the hash lines. It is not inconceivable that existing signature files could have been 
compromised in ways that might be missed when checking the signature. The total hash will point out such changes.

Examples:
> python signduterre.py --execute --detail --salt 436a73e3 --passphrase liauwefa3251EWC signduterre.py /sbin/* /bin/* \\
 /usr/bin/find /usr/bin/file /usr/bin/python* '${PATH}' > Signature_`date "+%Y%m%d_%H-%M-%S"`.txt

Prints a signature to the file Signature_20090625_14-31-54.txt (with your date). The signature contains the
SHA-256 hashes of the files, signduterre.py, /sbin/*, /bin/*,  /usr/bin/find, /usr/bin/file, /usr/bin/python*,
and a hash of the PATH environment variable.

> python signduterre.py --execute --detail --salt SUGGESTED --passphrase SUGGESTED --Status --detailed-view \\
signduterre.py /sbin/* /bin/* /usr/bin/find /usr/bin/file /usr/bin/python* '${PATH}' \\
2> Signature_`date "+%Y%m%d_%H-%M-%S"`.pwd > Signature_`date "+%Y%m%d_%H-%M-%S"`.txt

Prints a signature to the system Signature_20090625_14-31-54.txt (with your date) and the automatically generated
password to Signature_20090625_14-31-54.pwd (with your date). The salt will be automatically determined. 
The signature contains the SHA-256 hashes of the file status and file contents of signduterre.py, /sbin/*, 
/bin/*,  /usr/bin/find, /usr/bin/file, /usr/bin/python* on separate lines, and a hash of the PATH environment variable.

> python signduterre.py --execute --passphrase liauwefa3251EWC -c Signature_20090625_14-31-54.txt

Will check the files and PATH variable from the signature file Signature_20090625_14-31-54.txt.

> python signduterre.py --passphrase liauwefa3251EWC -c Signature_20090625_14-31-54.txt
> python signduterre.py --passphrase liauwefa3251EWC -c Signature_20090625_14-31-54.txt --no-execute

Will both fail if Signature_20090625_14-31-54.txt contains a $(cmd) entry. The --no-execute
option is default and prevents the execute option (if reading the execute optionfrom the signature file
has been activated).

> python signduterre.py signduterre.py --salt SUGGESTED -passphrase SUGGESTED=20 signduterre.py \\
&> Signature_`date "+%Y%m%d_%H-%M-%S"`.txt

Will generate and print 20 passphrases and print a signature using one randomly chosen passphrase
from these 20. Everything is written to a single file 'Signature_20090630_16-44-34.txt'.
On some systems, this means that the passphrases are written in between the hashes messing up the
file names. This will obviously make the output file useless.

> python signduterre.py signduterre.py -c Signature_20090630_16-44-34.txt

Will check all 20 passphrases generated before from the Signature file and print the results.

> sudo python signduterre.py  -u root -s SUGGESTED -p SUGGESTED -v -e -t \\
'$(ls -LCid /proc/$0/root /proc/$0/exe|sed "s|/$0/|/PID/|g")' '$(dd if=/dev/hda bs=512 count=1 | od -X)' \\
>Signature_`date "+%Y%m%d_%H-%M-%S"`.txt

Will hash the inode numbers of the effective root directory (eg, chroot) and the executable (python) 
together with the contents of the MBR (Master Boot Record) in Hex. It uses suggested salt and passphrase.
Accessing /dev/hda is only possible when root, so the command is entered with sudo and --user root.

"""
# 
license = """
Signature-du-Terroir
Construct a signature of the installed software state or check a previously made signature.

copyright 2009, R.J.J.H. van Son

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.
""";
#
# Note that only release notes are put here
# See git repository for detailed change comments:
# git clone git://repo.or.cz/signduterre.git
releasenotes = """
20090702 - Replaced -g with -p SUGGESTED[=N]
20090702 - Generating and testing lists of random salts
20090701 - Release v0.2
20090630 - Generating and testing random passphrases
20090630 - --execute works on $(cmd) only, nlinks in ?path and ? implied for directories
20090630 - Ported to Python 3.0

20090628 - Release v0.1b
20090628 - Added release-notes

20090626 - Release v0.1a
20090626 - Initial commit to Git
""";

import sys;
import os;
import stat;
import subprocess;
# if sys.stdout.isatty(): import readline;
import binascii;
import hashlib;
import re;
import time;
from optparse import OptionParser;
import base64;
import random;
import struct;

# Limit the characters that can be used in $(cmd) commands
not_allowed_chars = re.compile('[^\w\ \.\/\"\|\;\,\-\$\[\]\{\}\(\)\@\`\!\*]');

programname = "Signature-du-Terroir";
version = "0.2";

print("# Program: "+programname + " version " + version + "\n");

parser = OptionParser()
parser.add_option("-s", "--salt", metavar="HEX", 
                                  dest="salt", default=False,
                  help="Enter salt in cleartext. If not given, a hexadecimal salt will be suggested. The SUGGESTED[=N] keyword will cause the selection of the suggested string. N is the number of salts generated (default N=1)")
parser.add_option("-p", "--passphrase", metavar="TEXT",
                  dest="passphrase", default=False,
                  help="Enter passphrase in cleartext, the keyword SUGGESTED will cause the suggested passphrase to be used. Entering the name of an existing file will cause it to be read and a random passphrase found in the file will be used (creating a signature), or they will all be used in sequence (--check-file).")
parser.add_option("-c", "--check-file",
                  dest="check", default=False, metavar="FILE", 
                  help="Check contents of file (output of previous run)")
parser.add_option("-i", "--input-file",
                  dest="input", default=False, metavar="FILE", 
                  help="Use names from input-file (one filename per line)")
parser.add_option("-u", "--user",
                  dest="user", default="nobody", metavar="USER", 
                  help="Execute $(cmd) as USER, default 'nobody' (root/sudo only)")
parser.add_option("-S", "--Status",
                  dest="status", default=False, action="store_true", 
                  help="For each file, add a line with unvarying file status information: st_mode, st_ino, st_dev, st_uid, st_gid, and st_size (like the '?' prefix, default False)")
parser.add_option("-t", "--total-only",
                  dest="total", default=False, action="store_true", 
                  help="Only print the total hash, must be checked BEFORE running --detail (default True)")
parser.add_option("-d", "--detailed-view",
                  dest="detail", default=False, action="store_true", 
                  help="Print hashes of individual files, must be checked AFTER running --total-only check (default False)")
parser.add_option("-e", "--execute",
                  dest="execute", default=False, action="store_true", 
                  help="Interpret $(cmd) (default False)")
parser.add_option("-n", "--no-execute",
                  dest="noexecute", default=False, action="store_true", 
                  help="Explicitely do NOT Interpret ${ENV} and $(cmd)")
parser.add_option("-m", "--manual",
                  dest="manual", default=False, action="store_true", 
                  help="Print the manual and exit")
parser.add_option("-r", "--release-notes",
                  dest="releasenotes", default=False, action="store_true", 
                  help="Print the release notes and exit")
parser.add_option("-l", "--license",
                  dest="license", default=False, action="store_true", 
                  help="Print license text and exit")
parser.add_option("-v", "--verbose",
                  dest="verbose", default=False, action="store_true", 
                  help="Print more information")

(options, check_filenames) = parser.parse_args();
# Print license
if options.license:
        print (license, file=sys.stderr);
        exit(0);
# Print manual
if options.manual:
        print (manual, file=sys.stderr);
        exit(0);
# Print manual
if options.releasenotes:
        print ("Version: "+version, file=sys.stderr);
        print (releasenotes, file=sys.stderr);
        exit(0);

my_salt = options.salt;
my_passphrase = options.passphrase;
my_check = options.check;
my_status = options.status;
my_verbose = options.verbose;
execute = options.execute;
noexecute = options.noexecute;
input_file = options.input;

# Set total-only with the correct default
total_only = True;
total_only = not options.detail;
if options.total: total_only = options.total;
if my_check: total_only = False;

my_user = options.user;
# Things might be executed as another user
user_change = '';
if os.getuid() == 0:
        user_change = 'sudo -H -u '+my_user+' ';
        print("User: "+my_user);

# Execute option
if execute:
        text_execute = "True";
else:
        text_execute = "False";

if execute: print("Execute system commands: "+text_execute+"\n");

# Measure time intervals
start_time = time.time();

# Read input-file
if input_file:
        with open(input_file, 'r') as i:
                for line in i:
                        # Clean up filename
                        current_filename = re.sub('[^\w\-\.\/\$\{\(\)\}]', '', line);
                        check_filenames.append(current_filename);
                        
stat_list = [];
for x in check_filenames:
        if os.path.isdir(x):
                x = '?'+x;
        if my_status and not x.startswith(('?', '$')):
                stat_list.append('?'+x);
        stat_list.append(x);
check_filenames = stat_list;

# Read the check file
passphrase_list = [];
salt_list = [];
check_hashes = {};
total_hash = "";
if my_check:
        print("# Checking: "+my_check+"\n");
        check_filenames = [];
        with open(my_check, 'r') as c:
                for line in c:
                        match = re.search("Execute system commands:\s+(True|False)", line);
                        if match != None:
                                # Uncomment the next line if you want automatic --execute from the check-file (DANGEROUS)
                                # execute = match.group(1).upper() == 'TRUE';
                                continue;
                                
                        match = re.search("Salt\:\s+\'([\w]*)\'", line);
                        if match != None:
                                salt_list.append(match.group(1));
                                continue;
                                
                        match = re.search("User\:\s+\'([\w]*)\'", line);
                        if match != None:
                                # Uncomment the next line if you want automatic --user from the check-file (DANGEROUS)
                                # my_user = match.group(1);
                                continue;
                                
                        match = re.search("Passphrase\:\s+\'([^\']*)\'", line);
                        if match != None:
                                passphrase_list.append(match.group(1));
                                continue;
                                
                        match = re.search("^\s*([a-f0-9]+)\s+\*(TOTAL HASH)\s*$", line)
                        if match != None:
                                total_hash = match.group(1);
                                continue;
                                
                        match = re.search("^\s*([a-f0-9\-]+)\s+\*(.*)\s*$", line)
                        if match != None:
                                check_filenames.append(match.group(2));
                                check_hashes[match.group(2)] = match.group(1);
                                continue;

# Seed Pseudo Random Number Generator
dev_random = open("/dev/urandom", 'rb');
seed = dev_random.read(16);
dev_random.close;
random.seed(seed);

# Read suggested salts from /dev/(u)random if needed
if my_salt:
        if my_salt.startswith('SUGGESTED'):
                dev_random = open("/dev/urandom", 'rb');
                N=1;
                match = re.search("([0-9][0-9]*)$", my_salt);
                if match != None:
                        N = int(match.group(1));
                for i in range(0,N):
                        salt = dev_random.read(8);
                        salt_list.append(str(binascii.hexlify(salt), 'ascii'));
                dev_random.close;
        else:
                salt_list.append(my_salt);
elif len(salt_list) == 0:
        dev_random = open("/dev/random", 'rb');
        salt = dev_random.read(8);
        dev_random.close;
        sys.stderr.write("Enter salt (suggest \'"+str(binascii.hexlify(salt), 'ascii')+"\'): ");
        salt_list.append(input());

for my_salt in salt_list:
        print("Salt: \'"+my_salt+"\'");

# Get passphrase
if my_passphrase and os.path.isfile(my_passphrase):
    with open(my_passphrase, 'r') as file:
        for line in file:
                match = re.search("Passphrase\:\s+\'([^\']*)\'", line);
                if match != None:
                        passphrase_list.append(match.group(1));
elif not my_passphrase and len(passphrase_list) == 0:
        dev_random = open("/dev/random", 'rb');
        suggest_passphrase = dev_random.read(16);
        dev_random.close;
        sys.stderr.write("Enter passphrase (suggest \'"+str(base64.b64encode(suggest_passphrase), 'ascii').rstrip('=')+"\'): ");
        # How kan we make this unreadable on input?
        my_passphrase = input();
elif my_passphrase.startswith('SUGGESTED'):
    N = 1;
    match = re.search("([0-9][0-9]*)$", my_passphrase);
    if match != None:
        N = int(match.group(1));
        # Unblocked /dev/urandom is used here because /dev/random blocked
    dev_random = open("/dev/urandom", 'rb');
    j = int(random.random()*N);
    for i in range(0, N):
            suggest_passphrase = dev_random.read(16);
            current_passphrase = str(base64.b64encode(suggest_passphrase), 'ascii').rstrip('=');
            print("Passphrase: \'"+current_passphrase+"\'", file=sys.stderr);
            passphrase_list.append(current_passphrase);
    dev_random.close;
else:
    passphrase_list.append(my_passphrase);

if not my_check:
    j = int(random.random()*len(passphrase_list));
    passphrase_list = [passphrase_list[j]];
    print("# Selected passphrase:", j+1, file=sys.stderr);
    j = int(random.random()*len(salt_list));
    salt_list = [salt_list[j]];
    print("# Selected salt:", j+1, file=sys.stderr);

end_time = time.time();
print("# Preparation time:", end_time - start_time, "seconds\n\n# Start signature");

pnum = 1;
snum = 1;
corrpnum = 0;
corrsnum = 0;
for my_passphrase in passphrase_list:
        snum = 1;
        for my_salt in salt_list:
                start_time = time.time();
                # Construct the passphrase hash
                passphrase = hashlib.sha256();
                
                passphrase.update(bytes(my_salt, encoding='ascii'));
                passphrase.update(bytes(my_passphrase, encoding='ascii'));
                
                # Create prefix which is a hash of the salt+passphrase
                prefix = passphrase.hexdigest();
                
                # Create signature and write output
                if noexecute: execute = False;          # Doubly make sure that NOTHING is executed if required
                totalhash = hashlib.sha256();
                totalhash.update(bytes(prefix, encoding='ascii'));
                for filename in check_filenames:
                        # Create file hash object
                        filehash = hashlib.sha256();
                        filehash.update(bytes(prefix, encoding='ascii'));
                        # Use system variables and commands
                        if filename.startswith('$'):
                                # Commands $(command)
                                match = re.search('^\$([\(\{]?)([^\)\}]+)[\)\}]?$', filename);
                                if match != None:
                                        if match.group(1) == '(':
                                                if not execute : 
                                                        error_message = "Executable argument \'"+filename+"\' only allowed with the --execute flag";
                                                        print (error_message, file=sys.stderr);
                                                        if not sys.stdout.isatty(): print(error_message);
                                                        exit(1);
                                                current_command = not_allowed_chars.sub(" ", match.group(2));
                                                current_command_line = user_change+"bash --restricted -c \'"+current_command+"\' "+str(os.getpid());
                                                if my_verbose:
                                                        print ("#", current_command_line);
                                                (status, b) = subprocess.getstatusoutput(current_command_line);
                                                if status != 0: 
                                                        print ('$('+current_command+')'+"\n"+b, file=sys.stderr);
                                                        exit(status);
                                        else:
                                                current_var = not_allowed_chars.sub(" ", match.group(2));
                                                if my_verbose:
                                                        print ("# echo $"+ current_var);
                                                b = os.environ[current_var];
                                filehash.update(bytes(bytes(b, encoding='utf8')));
                        # lstat() meta information
                        elif filename.startswith('?'):
                                filestat = os.lstat(filename.lstrip('?'));
                                b = 'lstat('+filename.lstrip('?')+') = [st_mode='+str(oct(filestat.st_mode))+', st_ino='+str(filestat.st_ino)+', st_dev='+str(filestat.st_dev)+', st_nlink='+str(filestat.st_nlink)+', st_uid='+str(filestat.st_uid)+', st_gid='+str(filestat.st_gid)+', st_size='+str(filestat.st_size)+']';
                                filehash.update(bytes(b, encoding='utf8'));
                                if my_verbose:
                                        print ("# "+ b);
                        # Use file
                        else:
                                # open and read the file
                                with open(filename, 'rb') as file:
                                        for b in file:
                                                filehash.update(b);
                                                
                        current_digest = filehash.hexdigest();
                        current_hash_line = current_digest+" *"+filename
                        totalhash.update(bytes(current_hash_line, encoding='ascii'));
                        
                        # Be careful to use this ONLY after totalhash has been updated!
                        if total_only: current_hash_line = (len(current_digest)*'-')+" *"+filename;
                        
                        # Write output
                        if not my_check:
                                print(current_hash_line);
                        elif check_hashes[filename] ==  (len(current_digest)*'-'):
                                print(check_hashes[filename]+" *"+filename);
                        elif current_digest != check_hashes[filename]:
                                print("DIFFERENT: "+current_hash_line);
                        else:
                                print("ok"+" *"+filename);
                
                # Handle total hash
                current_total_digest = totalhash.hexdigest();
                current_total_digest_line = current_total_digest+" *"+"TOTAL HASH";
                end_time = time.time();
                print("# \n# Total hash - Time to completion:", end_time - start_time, "seconds");
                if not my_check:
                        print(current_total_digest_line+"\n");
                elif current_total_digest != total_hash:
                        print("DIFFERENT: "+current_total_digest_line+"\n");
                else:
                        match_number = "";
                        if len(passphrase_list) > 1 or len(salt_list): match_number = " #"
                        if len(passphrase_list) > 1: match_number += " passphrase no: "+str(pnum);
                        if len(salt_list) > 1: match_number += " salt no: "+str(snum);
                        print("OK"+" *"+"TOTAL HASH"+match_number+"\n");
                        corrsnum = snum;
                        corrpnum = pnum;
                snum += 1;
        pnum += 1;
        
if len(passphrase_list) > 1:
        if corrpnum > 0:
                print("Passphrase entry:",corrpnum,"matched");
        else:
                print("No passphrase entry matched!");
if len(salt_list) > 1:
        if corrpnum > 0:
                if corrsnum > 0:
                        print("Salt entry:",corrsnum,"matched");
                else:
                        print("No salt entry matched!");
        else:
                print("No entry matched");