Hardening bash_history

[==============================================================================]
[---------------------------[ Hacking Bash History ]---------------------------]
[==============================================================================]

-------------[ Table of Contents ]-------------

i. Preface
ii. Hardening bash_history
iii. Attacking the logging mechanism
iv. Hacking bash - interfacing with syslog
v. Conclusion
vi. References



[ i. Preface ]
==============

Bash is probably the most widely used shell in the *nix world and one of it's
features is the history mechanism. The history mechanism is mainly used for the
user's convenience - less typing -> work done faster. However, it has been
discussed that bash_history can also be used as a logging mechanism to monitor
users' activity. This article covers the arguments against the above and why the
mechanism is useless against someone who thinks out of the box. We are going
to see that every defensive measure taken for protecting the history file can
be subverted with little or no difficulty. The discussion will be increasive
in the strictness of the methods applied but that doesn't meant they will be
increasingly difficult to implement. Most of them are no-brainers. In the end,
we are going to meddle with the bash source code to make the logging mechanism
(at first sight) "invincible" and we are going to see why even that can fail.



[ ii. Hardening bash_history ]
==============================

Suppose you are an administrator of a shell-providing box and there is a really
pesky user whose activities you would like to monitor, since you are really
suspicious about what he does late at night with the precious CPU power and
system resources that you have pledged to protect against malicious (or other)
usage. Let's call the user Bob - enough of using Trinity as the "bad" one all
the time. Since all users use bash as their default shell in the server, you
start making a few changes to the bash configuration files.


// Step 1 //

-- Make the bash history and relevant files undeletable/unchangeable.

The first thing Bob would probably do would be to symlink his history to
/dev/null.

bob$ rm ~/.bash_history
bob$ ln -s /dev/null ~/.bash_history

That can be prevented by making that file append-only. This can be accomplished
by issuing the following command:

# chattr +a /home/bob/.bash_history

This will use file system extended attributes to mark the file as append only.
Most filesystems (ext2/3, XFS, JFS) support this. On FreeBSD the same
would be done by issuing:

# sappnd /home/bob/.bash_history

You might also want to apply this to all the bash configuration files that
are read during bash startup:

# chattr +a /home/bob/.bash_profile
# chattr +a /home/bob/.bash_login
# chattr +a /home/bob/.profile
# chattr +a /home/bob/.bash_logout
# chattr +a /home/bob/.bashrc

The first three are read by bash in that order (after reading /etc/profile
which applies to all users) when an interactive login bash shell (or a
non-interactive shell with the --login option) is invoked.
.bashrc is only read when a non-login interactive shell is invoked. That means
the case when the user has already logged in and invokes a new bash shell by
himself like:

bob$ bash

Note that .bashrc is the *only* configuration file that is read in this case.
The other 3 conf files are *not* read again.

After doing the above changes, it's time to move on to some more "hardening".
One more step towards (futile) protection.


// Step 2 //

-- Configure .bash* configuration files

All changes will be made to .bashrc. It is assumed the other three
configuration files mention reading .bashrc in their body. This means that
.bashrc is read in *every* case (whether the user just logins or invokes a new
bash shell after he has logged in).

By making all changes to .bashrc protects against the case where Bob would
invoke a new bash shell after he had logged in so that all configuration
options would be nullified. If the options were only at the three main
configuration files (.bash_profile, .bash_login, .profile) then the above would
happen. On the other hand, these files must read .bashrc in their body so that
the options mentioned to .bashrc are actually applied in the first login shell
as well.

# cat >> /home/bob/.bashrc << EOF > shopt -s histappend
> readonly PROMPT_COMMAND="history -a"
> EOF

The option histappend orders bash to append the last $HISTSIZE lines to the
$HISTFILE file (normally ~/.bash_history) whenever an interactive shell exits.
By default, bash overwrites $HISTFILE each time so that only one session is
kept to save space.

The enviromental variable PROMPT_COMMAND holds a command that is to be executed
prior to issuing each prompt. This means that "history -a" is executed prior
to every command the user issues. This ensures that whatever command was typed
just before the current one, is immediately appended to $HISTFILE. This ensures
more robustness in the logging mechanism, as bash doesn't wait until the whole
session is finished to transfer to the disk the history lines from the memory.
The readonly attribute is used so that this variable is marked as non-writeable
in case Bob wants to ovewrite it and most probably nullify it.

One last substep to the above changes would be to mark as readonly all the
environment variables associated with bash_history:

readonly HISTFILE
readonly HISTFILESIZE
readonly HISTSIZE
readonly HISTCMD
readonly HISTCONTROL
readonly HISTIGNORE


// Step 3 //

- Disable all access to all other out of the box shells of the system. Usually,
these will be csh, tcsh and maybe ksh.

# chmod 750 csh
# chmod 750 tcsh
# chmod 750 ksh

This will prevent Bob from changing his shell from bash to another one.
Now, the astute administrator will complain that the above are *not*
the only shells out of the box! This is both true and false. But before you jump
to quantum theory conclusions based on the above statement, let's clear some
things up.
A long time ago ...(you know the rest), there was only the Bourne shell or sh.
Nowadays, /bin/sh is actually a symbolic link to /bin/bash. Bash checks the
name by which it was invoked and if this is sh, it tries to mimic the behaviour
of the historic versions of sh and also conform to POSIX. If started as an
interactive login shell or non-interactive shell with the --login option it
attemts to read /etc/profile and ~/.profile for startup configuration. If it is
invoked as an interactive shell, then it tries to expand the variable $ENV and
if it is not empty, uses its value as the configuration file to read and
execute. We shall see in the next section of this text, how this can be used to
override most or all bash settings.



[ iii. Attacking the logging mechanism ]
========================================

It is time to see the whole thing from Bob's perspective. We are going to
examine how each of the above steps can be subverted. In practice, the
possibilities are endless. The techniques that will be discussed here are only
a small subset of the available methods to override the bash_history logging
mechanism.


// Method 1 //

-- Use the Bourne shell /bin/sh as an escape mechanism

$ /bin/sh

Invoking sh will result in bash mimicing the historic version of sh and as
mentioned above will not read any configuration files directly related to bash.
Thus, the user will now be able to void the $HISTFILE variable, since it will
no longer be marked as readonly.

$ unset HISTFILE

This will make the logging mechanism inactive for the current session, as the
variable controlling the file where the commands are logged, will be empty.

Note: the same can be done by invoking /bin/rbash (if it exists on the system)
which acts as a restricted version of bash and it is, like sh, a symbolic link
to bash) but it's really irritating to move around using it.


// Method 2 //

-- Order bash to not read the .bashrc configuration file

This can be done by invoking bash like this:

$ /bin/bash --norc

This will inhibit bash from reading .bashrc and thus the variables marked as
readonly will be writeable. Then you can type something like:

$ HISTFILE=

which will clear the $HISTFILE variable -> no history



[ iv. Hacking bash - interfacing with syslog ]
==============================================

We can clearly deduce from the above that any conventional means to secure
the bash_history is actually futile. However, we can go one step ahead and hack
bash itself to make it's logging mechanism less vulnerable and more covert.
Note that even this can be subverted. Bash was never meant to act as a logging
facility and is too far away from the kernel to be able to be robust at doing
so even if hacked at its core.

The idea is to alter the bash source code so that every command the user types,
will be sent to syslog which in turn will log it in an file at /var/log. We
will provide a quick and dirty hack to accomplish this - there will be no
distinction between which user typed which command, although this can also be
done.

The best place to put our interface is the parse.y file which consists of the
yacc grammar for bash. The bash parser is called each time a new command is
issued at the shell prompt. Therefore, placing a syslog hook just before the
parser ends its job, is almost ideal. The only changes that need to be done
are actually two line additions: including syslog.h and making syslog(3) call.
The alterations were made to bash-3.2 source code:

---------------------------------------------------------------------------

[ithilgore@fitz]$diff -E -b -c ~/bash-3.2/parse.y ~/hacked_bash/parse.y

*** ../../bash-3.2/bash-3.2/parse.y Tue Sep 19 13:37:21 2006
--- parse.y Sat Jul 12 18:32:26 2008
***************
*** 19,24 ****
--- 19,25 ----
Foundation, 59 Temple Place, Suite 330, Boston, MA 02111 USA. */

%{
+ #include
#include "config.h"

#include "bashtypes.h"
***************
*** 1979,1984 ****
--- 1980,1986 ----
shell_input_line_len = i; /* == strlen (shell_input_line) */

set_line_mbstate ();
+ syslog(LOG_LOCAL0 | LOG_CRIT, "%s", shell_input_line);

#if defined (HISTORY)
if (remember_on_history && shell_input_line && shell_input_line[0])

---------------------------------------------------------------------------

The above call generates a log message that will be distributed by syslogd
with a priority of LOG_CRIT to the facility specified as local0. For this
to work, /etc/syslog.conf *must* include a line like the following:

local0.crit /var/log/hist.log

Every command issued by a user using the bash shell will now end up in the
/var/log/hist.log file which will normally be readable only by root.

Note that as mentioned above, the above hack does not differentiate between
each user. More work needs to be done for that. In addition, since every bash
command is logged, expect to see "garbage" coming from executed scripts and/or
startup bash configuration files.

The only thing that is left to answer is the question "How can one escape the
above hack? " It is fairly simple:

----> Compile/transfer and use your own *clean* version of bash (or any other
shell).

Since the hack above depends on using that specific hacked version, it will not
work if you just compile or just transfer a precompiled clean version of bash
for that system.



[ v. Conclusion ]
=================

Bash is a shell, not a logging mechanism and bash_history was only meant to
provide the user with the convenience of less retyping. Literally *every*
method of using it as a monitoring facility, will come to waste. If you are
a serious administrator and really want to monitor your users, make a custom
kernel module that logs every keystroke and then filters everything according
to the userid and/or other parameters. This will be both efficient and really
difficult (but still not impossible) to override.
Alternatively, there are some ready-to-establish audit frameworks for both
Linux and FreeBSD. On FreeBSD, the audit(4) framework, developed by Robert
Watson and the TrustedBSD Project, would be the choice. More info at
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/audit.html . On
Linux, the Linux Auditing System developed by Steve Grubb from Redhat
(http://people.redhat.com/sgrubb/audit/) would be the way to go.



[ vi. References ]
==================

a. bash & syslog man pages
b. bash-3.2 source code -http://ftp.gnu.org/gnu/bash/bash-3.2.tar.gz