Hardening Solaris 

Secure installation of Bastion hosts

By Seán Boran
www.boran.com/security/sp/Solaris_hardening2.html

Note:
- a newer article is available in draft form and includes Yassp beta#11.
- This version will not longer be updated.

This article presents a concise step-by-step approach to securely installing Solaris for use in a firewall DMZ or other sensitive environment, using the Yassp tool (beta#5) and (with Solaris 8) the Sunscreen EFS firewall.

The focus in this article is on preparing the Operating System to securely run services, but tips are also provided on the use of free security tools and configuring common services.

This article was written (and tested) for Solaris 2.6/7/8, but the techniques described here should work fine on earlier versions too. This article has been updated since the original release, see the section Changes to this article.

Your feedback is welcome.

The hardening process is divided into the following steps:

1. Preparation
2. Initial OS installation
3. Install SSH
4. Run YASSP
5. Solaris 8: Install Sunscreen EFS
6. Installing tools & sysadmin software
7. Logging
8. Install an Integrity Checker: e.g. Tripwire
9. Install, test, harden applications: General, DNS, ftp, httpd, smtp
10. Going Live

Regular maintenance
Yassp Overview
Additional Notes
References
Changes to this article

1. Preparation

• Keep things simple: it is expected that only one or two services will run on a host. Use several machines, rather than one superserver that does everything. It's easier to isolate applications, harden, troubleshoot and upgrade hw/sw. Be minimalist, only run what is absolutely necessary.
• Hardware: Consider installation via the serial port console, get rid of the keyboard, screen and framebuffer. i.e. avoid using X11 and get to know the command line. Have an isolated, trusted network available for testing.
• Downloading securely: Hopefully the installation is on an isolated or non-routed private network, in this case you can ftp to the new box (as root), or ftp from the new box (via the console) to a server on this net. If you don't have an isolated network (not advisable), then change the root password to something easy, download the files and change the password again, shutdown the network interface immediately. This reduces the window of opportunity for potential attackers. Alternatively setup a non-privileged account with /tmp as it's home and use to exchange files (Solaris 8 blocks ftpd for root by default).
• Know exactly what the system is supposed to do, what it's hardware configuration will be etc. hardening is generic and may break certain functions. e.g. CDE/OpenWindows, Disksuite and Legato need RPC to run but you really don't want RPC running on a firewall host, see also the notes on RPC below.
• It's important to understand how the applications work (how they use ports, devices, files), to judge what hardening is possible and to assess the risk posed.

2. Initial OS installation

Connect the serial console, switch on, halt to the OK prompt by sending a Stop-A (~#, ~%b, or F5 depending on whether you use tip, cu or a vt100 terminal), then start the installation procedure:  boot cdrom - install .
Note: On Solaris 8, make sure your use "Software CD#1" and not the "Installation CD".

Install the end user bundle (or even better only the core packages), set hostname, terminal, IP parameters, timezone, etc. Don't enable any naming services like NIS or NFS. Don't enable power management, or mount any remote file systems (NFS).
Note: On Solaris 8, the user bundle can be customised via "F4", packages can be added or removed, e.g. for a web server Apache can be added.

Choose manual disk partitioning:

• Keep /usr and /opt separate from root, so that they can be later mounted read-only (see below). Consider a separate, large /var filesystem for syslog/web/news/proxy servers or firewall filters. Servers containing lots of data (web, ftp) should use a separate disk for their data.
• If you don't wish to mount partitions read-only, then consider putting the whole boot disk under root.
• Suggestion for a 2GB disk: 200MB / (root+var), 200MB swap,  600MB /usr and 1GB on /opt.
• Suggestion for a 1GB disk: 300MB / (root+var+opt), 200MB swap,  500MB /usr.

Set a strong password (7 or 8 chars with numbers, letters and punctuation) for root.

Reboot. 

If you use the C-Shell, the following will make editing easier and enable history functions:

csh;
setenv TERM vt100; setenv VISUAL vi; setenv EDITOR vi;
set filec; set history=40; alias h history; alias ls 'ls -aF \!*';

No man pages are installed with the user bundle, so install them while the Solaris CD is still mounted:

cd /cdrom/cdrom0/s0/Solaris_2.6/Product;
pkgadd -d . SUNWman SUNWdtma SUNWjvman SUNWpmowm SUNWolman SUNWtltkm SUNWxwman SUNWxwpmn

cd /cdrom/cdrom0/s0/Solaris_2.7/Product;
pkgadd -d . SUNWman SUNWdtmaz SUNWdtma SUNWjvman SUNWpmowm SUNWolman SUNWtltkm SUNWxwman SUNWxwpmn

On Solaris 8, insert CD#2:
cd /cdrom/cdrom0/Solaris_8/Product;
pkgadd -d . SUNWman SUNWdtmaz SUNWdtma SUNWjvman SUNWpmowm SUNWolman SUNWtltkm SUNWxwman SUNWxwpmn SUNWj2man

Update indices, so that "man -k keyword" will allow searching of relevant man pages:

/usr/lib/makewhatis /usr/man;
/usr/lib/makewhatis /usr/openwin/man;

If a patch CD is available, install the recommended & security patches from Sun. Typically a separate CD "Maintenance Release" is provided with a patch bundle.
Reboot and logon again as root. The list of patches installed can be listed with showrev -p .
Note that the current patches will be checked and updated over the network in Step 6 below.

3. Install SSH

Install SSH, the secure Shell for login access (to avoid using telnet/ftp). There are two key implementations 'ssh1' and 'OpenSSH', here we use ssh1 as an example. OpenSSH is more interesting in some ways (truly free and support for ssh2 protocol), but ssh1 also supports securid (which is useful to me) and is easier to install. OpenSSH will probably become the definitive standard in the future. See also [7] for a detailed discussion of SSH and it's various implementations.

Download sources (ssh-1.2.27.tar.gz from [7a]) and compile on a separate machine

zcat ssh-1.2.27.tar.gz | tar xf - 
cd ssh-1.2.27; ./configure --prefix=/usr --without-none --without-rsh --without-idea
make

and create a binary distribution. See also [7] for a detailed discussion of SSH and it's various implementations.

Download ssh_bin.tar.Z (or whatever your binary is called) to the new target system:
extract in root, "rehash" (if using csh) and then generate a host key:
     zcat ssh_bin.tar.Z | tar xvf -;
   /usr/local/bin/ssh-keygen -b 1024 -f /etc/ssh_host_key -N '';
Add the ssh service to /etc/services:
     ssh 22/tcp            # Secure Shell
Start the ssh daemon:
     sh /etc/rc2.d/S10sshd start

Configure an appropriate /etc/ssh_config file (see also [7]), so that access is restricted to named hosts with known public keys (/etc/ssh_known_hosts) and rhosts authentication is disabled. Avoid trusts. Only allow specific users and hosts to access SSH. Deny daemon accounts access.

4. Run YASSP

Jean Chouanard has developed the Yassp (Yet Another Solaris Security Package) scripts for hardening Solaris and which have evolved quit a bit this year. See the Yassp section below which presents Yassp in more detail. In particular we use the 'cleanup' and 'SECclean' parts of Yassp. We use Yassp for this section of the hardening because it is comprehensive, is a consensus of security experts, can be backed out, has been tested by many people and is backed by SANS. An example of yassp output is useful in understanding what it does: yassp_output.txt.

Download clean-up_5.X.tar.Z and secclean.Z [1] to the target.

Shutdown the network interface during this next phase, just in case (the interface name depends on the architecture e.g. hme0 on UltraSPARC):

ifconfig le0 down

Run the package content database cleaning and file permission tightening:

zcat clean-up_5.7.tar.Z | tar xvf -
/var/sadm/clean-up_5.7/install.sh

There should be no errors and the output of the last command "pkgchk -n" should be null, except a warning on "/home" as the automounter is still running. It make take 15 minutes.. 
If this step needs to be undone, run /var/sadm/clean-up_5.7/un-install.sh

Run the hardening package (make take 15 minutes):

uncompress secclean.Z;
pkgadd -d secclean SECclean

Note: Changed files are backed up under /var/sadm/pkg/SECclean/save.

Remove scripts:

rm clean-up_5.7.tar.Z secclean

Reboot.

Login as root and check the process list, it should be something like:
  tests# ps -ef
UID PID PPID C STIME TTY TIME CMD
root 0 0 0 15:13:51 ? 0:01 sched
root 1 0 0 15:13:54 ? 0:00 /etc/init -
root 2 0 0 15:13:54 ? 0:00 pageout
root 3 0 0 15:13:54 ? 0:00 fsflush
root 170 167 0 15:14:16 ? 0:00 /usr/lib/saf/ttymon
root 138 1 0  15:14:14 ? 0:00 /usr/sbin/syslogd
root 167 1 0  15:14:15 ? 0:00 /usr/lib/saf/sac -t 300
root 168 1 1  15:14:15 console 0:00 -sh
root 152 1 0  15:14:15 ? 0:00 /usr/lib/utmpd
root 174 168 1 15:14:56 console 0:00 csh
root 137 1 0  15:14:14 ? 0:00 /usr/sbin/cron
root 175 174 1 15:14:58 console 0:00 ps -ef

check the network connections, only syslog should be listening:

UDP
Local Address Remote Address State
-------------------- -------------------- -------
*.syslog Idle
*.* Unbound

TCP
Local Address Remote Address Swind Send-Q Rwind Recv-Q State
-------------------- -------------------- ----- ------ ----- ------ -------
*.* *.* 0 0 0 0 IDLE
*.* *.* 0 0 0 0 IDLE

Solaris 8: Fixes for issues not yet supported by Yassp beta#3:

• /etc/init.d/inetsvc is not properly replaced. If you don't need DHCP, DNS or inetd it doesn't matter. If you do, then:
  cp /etc/init.d/inetsvc.SUN_Before_YASSP /etc/init.d/inetsvc
  then edit it: add '-t' to inetd command line, comment out DHCP if not needed.
• inetinit is not properly replaced. Copy it back
  cp /etc/init.d/inetinit.SUN_Before_YASSP /etc/init.d/inetinit
  It contains lots of new IPv6/IPsec stuff. Make sure /etc/notrouter exists to ensure no routing daemons are run, and move the routing binary:
  mv /usr/sbin/in.routed /usr/sbin/in.routed.NOTUSED
• One of the tune up parameters it not recognised on Solaris 8, so disable the following line in /etc/init.d/nettune:
  ndd -set /dev/ip ip_ire_flush_interval 120000
• Stop the cimomboot daemon, which is not documented in the man pages and it not stopped by Yassp:
  mv /etc/rc2.d/S90wbem /etc/rc2.d/.S90wbem

Configuring the hardening level - typically nothing needs to be enabled for bastion hosts. These files are well commented and easy to understand.

vi /etc/rc.conf
vi /etc/yassp.conf

If any user other than root needs to use at/cron, the allow files in /etc/cron.d will need adapting.

5. Solaris 8: Install Sunscreen EFS

Sunscreen EFS Firewall: On Solaris 8, the iPlanet CD#2 contains among other things, a restricted edition of the Sunscreen EFS firewall, 'lite v3.0.1'. It can be used to protect network communications to the local machine (among other things).

Below we present a quick overview of setting up some simple rules on the command line, see [9] for a detailed discussion of Sunscreen configuration.

• Installation: When installing on the test system (which was installed with the Solaris 8 end user bundle), the EFS installation complained that SUWsprot was not installed. So, put back in Solaris CD#2 and install it first:
  pkgadd -d  /cdrom/sol_8_sparc_2/Solaris_8/Product SUNWsprot
  
  Put back in the iPlanet CD#2, and start the GUI installation tool:
  /cdrom/cdrom0/SunScreen/screenInstaller
  For this example, we accept all defaults options except: Naming services=DNS (not NIS).
• Configuration: Now we setup some simple Firewall rules. First, find out the name of the current running firewall policy, edit it and list the default rules and addresses.

# cd /opt/SUNWicg/SunScreen/bin;
# ./ssadm active
Active configuration: www default Initial.2
# ./ssadm edit Initial
edit> list rule
1 "common" "*" "*" ALLOW
edit> list address
"*" RANGE 0.0.0.0 255.255.255.255
"le0.net" RANGE 176.17.17.0 176.17.17.255
"localhost" HOST
"smtp-server" HOST 1.1.1.1
"www_le0" GROUP { } { }
edit> list service common
"common" GROUP "tcp all" "udp all" "syslog" "dns" "rpc all" "nfs prog" "icmp all" "rip" "ftp" "rsh" "real audio" "pmap udp all" "pmap tcp all" "rpc tcp all" "nis" "archie" "traceroute" "ping"

So we see that the default policy allows quite a few services through.

Let's presume that we are setting up a HTTPD server (on port 80) and intend to manage it via SSH. We also want to allow ping and traceroute for initial trouble shooting. We could then create and active the new firewall policy restricting access to these services as follows

# ./ssadm edit Initial
edit> add service ssh      SINGLE FORWARD "tcp" PORT 22
edit> add service myhttp GROUP ping traceroute ssh www
edit> replace rule 1 ALLOW myhttp "*" "*"
edit> list rule
1 "myhttp" "*" "*" ALLOW
edit> save
edit> verify
Configuration verified successfully (not activated).
edit> quit
www# ./ssadm activate Initial
Configuration activated successfully on www.

Voila!

• In fact, we would probably only allow ssh, ping and traceroute to specific management hosts, to reduce the risks even further. We can also allow dns lookups.

# cd /opt/SUNWicg/SunScreen/bin;
# ./ssadm edit Initial
edit> add address mgt_net RANGE 176.17.17.0 176.17.17.255
edit> add service mgt GROUP ping traceroute ssh
edit> replace rule 1 ALLOW www "*" localhost
edit> replace rule 2 ALLOW mgt mgt_net localhost
edit> replace rule 3 ALLOW mgt localhost "*"
edit> replace rule 4 ALLOW dns localhost "*"
edit> save
edit> verify
Configuration verified successfully (not activated).
# ./ssadm activate Initial
Configuration activated successfully on www.

Now test the network connections, to ensure the rules have the desired effect. To rollback to the initial configuration, delete all rules and add:

replace rule 1 ALLOW "common" "*" "*"

Finally, we can stop the remote Firewall management GUI. Why would we do this? If we are comfortable with the command line "ssadm" then one daemon more and one more configuration interface, that needs to be correctly configured and watched.

Disabling the following line in /etc/rc2.d/S63sunscreen
$SS_LIBDIR/run_httpd start efshttpd

In /opt/SUNWicg/SunScreen/lib/ss_boot, disable:
$SS_LIBDIR/ssadmserver start >/dev/console 2>&1

See also [9] for a detailed discussion of Sunscreen configuration.

6. Installing tools & sysadmin software

The system has now gone through a first hardening phase and should be still working! Boot and login on the console as root. Check for error messages on the console, fix if necessary.

More Solaris patches: During installation, some patches were installed. However, it's unlikely that they are the newest and may have been only the recommended ones.. Now that the system is reasonably secured, it's time to get the most recent versions of security patches. Get the Patchdiag tool from Sunsolve along with and update patchdiag.xref [15] and run it to see what recommended and security patches are needed, then download & install the missing ones. If you don't have a support contract just installed the latest security bundle or use the SecurityFocus vulnerability calculator  [15].
DNS note: even if you install BIND, the Solaris libresolv still needs patching.

Environment:

/.cshrc /.profile: set aliases, variables (such as VISUAL, EDITOR and PATH don't include "."). Set umask to 077, or 027.

Search path: Yassp adds /opt/local/bin to the su path (in /etc/default/su), if your tools are more likely to be in /usr/local, this will need adapting.

Disk mounting (vfstab options):
Several options can be set to improve the security of robustness of filesystems when they are mounted.

• In /etc/inetd.conf, all services have been disabled by Yassp: reopen very specific services, only if absolutely needed.
• Configure /etc/hosts with a list of critical machines (which you don't want resolved via DNS).
• DNS client (avoid if not needed): add domain name & DNS servers to /etc/resolv.conf. Add a DNS entry for "hosts" in /etc/nsswitch.conf.
• Use default routes: add the IP address of the router to /etc/defaultrouter, or create a startup file in /etc/rc2.d/S99static_routes using the "route" command.

Email: If hosts are not supposed to send email outside the subnet, don't configure the mailhost alias. Delete /usr/lib/sendmail if you don't need any kind of email.
Otherwise edit /etc/mail/aliases (at least point root to a real address), set mailhost in /etc/hosts and in /etc/mail/sendmail.cf set the following to ensure all outgoing email is channeled over mailhost:
   Dj$w.YOURDOMAIN.COM.
   DSmailhost
   DRmailhost
   DHmailhost
   O FallbackMXhost=mailhost
Add a hostname.YOURDOMAIN.COM alias for this machine in /etc/hosts.
Send a test email to check the config:
   mailx -v -s test_email root </dev/null

Install tools and scripts: Tools should already have been compiled and tested extensively on another machine.

• Solaris 8: Lots of Goodies are provided with Solaris 8 on the many additional CDs, one is the The Software Companion CD which contains popular Freeware, such as ppp, samba, wu-ftp, Development/Libraries, Development/Tools, X11 apps, vim/emacs, window managers and corresponding sources.
  Some tools not listed, that I would have liked were: top, lsof, gzip. Apache is not listed, because is included with Solaris itself (CD #2). These tools may save you some time, but will of course not be the very newest versions. Sun is to be commended for this CD, it's a pity there's not more and it didn't come sooner!
  There is a GUI install tool, to use it on a headless server (i.e. you have no screen), login via SSH and tunnel X11. Or just navigate the directories and use pkgadd.
  Choose custom install to control exactly what is installed, where.
• Install other useful tools: gzip, traceroute (included in Solaris 7) and maybe top (without SUID) or lsof.
• Install perl  (included in Solaris 8) for scripting and create a link to /bin/perl.
• Install security scripts in /secure, for example those used below [3]: rotate_cron, rotate_log, wtrim.pl, rdistd. Protect /secure:  
     chmod -R 700 /secure; chown -R root /secure

7. Logging

Configure logging:

Disable the Solaris log pruning (& other) lines in the root cron, since you're doing it yourself:

#10 3 * * 0,4 /etc/cron.d/logchecker
#10 3 * * 0 /usr/lib/newsyslog 

Syslog logging: Split up log analysis according to the example /etc/syslog.conf [3] which enables more logging than the default and splits up services into separate logfiles. Designate one machine as the loghost (in /etc/hosts) or log locally.

Syslog "loghost": Give the loghost a whopping great disk for logs. Create empty logs & set permissions:

cd /var/log;
touch daemonlog authlog kernlog userlog maillog lprlog cronlog newslog local0log local2log local5log alertlog;
chmod 600 daemonlog authlog kernlog userlog maillog lprlog cronlog newslog local2log local5log alertlog;
kill -1 `pgrep syslogd`

Use rotate_log [3] to prune & compress logs, add the root cron entries:
## Prune syslog logs weekly, keeping the last 6 months or so:
55 23 * * 6 /secure/rotate_log -n 40 alertlog
55 23 * * 6 /secure/rotate_log -n 40 authlog
55 23 * * 6 /secure/rotate_log -n 20 cronlog
55 23 * * 6 /secure/rotate_log -n 40 daemonlog
55 23 * * 6 /secure/rotate_log -n 40 kernlog
55 23 * * 6 /secure/rotate_log -n 40 locallog
55 23 * * 6 /secure/rotate_log -n 20 newslog
55 23 * * 6 /secure/rotate_log -n 40 userlog
55 23 * * 6 /secure/rotate_log -n 10 lprlog
55 23 * * 6 /secure/rotate_log -n 20 maillog

Pruning of login logs:

## Empty login/logout records at year end
0 0 31 12 * /secure/wtrim.pl wtmp 20
0 0 31 12 * /secure/wtrim.pl wtmpx 20
# Solaris 2.x logs:
0 4 * * 6 /secure/rotate_log -L /var/adm -n 30 loginlog
0 4 * * 6 /secure/rotate_log -L /var/adm -n 30 sulog
0 4 * * 6 /secure/rotate_log -L /var/adm -n 2 vold.log
0 4 * * 6 /secure/rotate_cron

Note: I've written a script /etc/weekly [3] which includes the above weekly entries and can be run on both loghost and client, with a cron entry like this:
50 23 * * 6 sh /etc/weekly 2>&1| tee /var/log/weekly.out

Crons

Root cron entries:

Set date once a day with a reliable source using rdate (you may prefer NTP, it's more accurate, but complex, uses bandwidth and is an additional security worry):

## Synchronise the time:
0 * * * * /usr/bin/rdate YOURTIMEHOST  >/dev/null 2>&1

# process the mail queue hourly:
0 * * * * /usr/lib/sendmail -q

File permissions: tighten more permissions, and restrict certain tools to root or disable:

chmod 0500 /usr/sbin/snoop /usr/sbin/devinfo
chmod o-r /var/spool/cron/crontabs/*
chmod 000 /bin/rdist
chmod o-rx /etc/security
chmod og-rwx /var/adm/vold.log
chmod u-s /usr/lib/sendmail       #Not for mailgateways or multi-user hosts
chmod 400 /.shosts /etc/sshd_config /etc/ssh_known_hosts

Documentation:

Document configuration changes in a text file such as /etc/mods, update after each change, with date, author, files affected, description.
cat > /etc/mods <<EOF
10.4.00  sb  New install of Solaris8 & tools according to hardening guidelines
EOF

Edit login banners to warn users about unauthorised access, you'll need this if you want to prosecute intruders. For Telnet and SSH, use /etc/issue (for pre-login) and /etc/motd (for post-login). Default banners are installed by yassp.

Reboot, login via SSH.

• Test - Do SSH and the standard tools work? Check log entries, check console messages. Does the system behave as expected?
• When all is working fine, freeze /usr and if possible /opt:

Mount /usr and /opt read-only (in /etc/vfstab with "ro" option). This reduces the risk of trojan horses and unauthorised modifications.
Mount other partitions nosuid (SUID programs cannot assume other identities).
Reboot.
Run the mount command to check that filesystems options are effective.

8. Install an Integrity Checker: e.g. Tripwire

At this stage, we need to install a file integrity checker that uses secure hashing algorithms, initialise it's database and run regular checks to monitor for changes. If possible, keep the master database on another machine or offline or on write-once media.

What options do we have for integrity checking?

• Tripwire [5]: There are both free and commercial versions.
  • The free version can be tricky to get working correctly and has a few bugs. Source code is provided. It crashes on very large disks (for me).
  • The commercial version is a bit pricey, reports are too verbose (you'll need filter scripts - V2.2.1 is better in this regard, then again 2.2.1 is buggier than 2.0.1), more configuration examples should be provided. It is more stable than the free version, also runs on RH Linux and NT and offers enhanced security by cryptographic signing of policy and configuration files. Support (even when paid for) is not great.
  • Neither version supports the use of regular expressions when defining policy rules. e.g. you can't specify a rule for "/home/*/www/cgi-bin" files, that would work even when new directories are added under /home.
  • Apparently, Tripwire will be released as OpenSource for Linux (but not Solaris?) near then end of 2000.
  • A good mix is to use the commercial version on secured central host, which then runs the free version remotely via SSH on all other hosts (see trip_host.sh notes below).
• PGP can also be used, by signing files to be protected (creating lots of signature files), then writing a script to check the validity of signatures. This will not catch permission, link, inode or modify date changes though.
• MD5 signatures could also be used in a similar way, but the list of MD5 signatures should not be stored on the system being monitored, unless it is PGP signed or encrypted.
• mtree is the tool used on OpenBSD, perhaps it will be ported to Solaris ?
• Aide is a new GPL tripwire replacement [12] that looks interesting, I've not had a chance to test it so far.

An example using the free Tripwire Version 1.2:

• Pick up the sources and compile. (An example binary that runs from /secure is tripwire, and example configuration is tw.config.SunOS  [3])
• Run tripwire with the "-i 2" option to increase speed (it disabled one checking algorithm, snerfu, but SHA1 and MDS 5 are still used).
• After installation, the "initial state" of the system needs to be saved:
  ./tripwire -i 2 -initialise -c tw.config
  This will create a new file database.
• The checking can be run each day from cron, or manually via:
  ./tripwire -i 2 -c tw.config
• To tell Tripwire that (changes to) files or entire directory trees are OK:
  tripwire -update [/file1 /file2 /patch3....]
• For increased security and automated checking of several systems from one trusted host, copy tripwire & it's database and run it remotely at regular intervals using SSH. Delete the tripwire database on the target after checking. This makes it difficult for an attacker to know that tripwire is being used to check the system. In addition, immediately update the tripwire database, so that only differences are reported by successive runs. See the sample script for doing exactly that trip_host.sh.
• Regularly copy the configuration and database to floppy disk or write-one media, in a crisis it will be very useful.

9. Install, test, harden applications

General

Consider installing applications on a separate partition or /opt. If /opt is used, it will have to be mounted read-write during installation and testing, and switched back to read-only afterwards.

Depending on the function of the server, applications such as ftpd, BIND, proxies, etc. are installed at this point. Web servers and firewall engines in particular, are complex and require careful configuration. The following is a general checklist that servers/applications should conform to:

• Is the umask set restrictively (e.g. 022) by or before the application starts?
• Does it run as a non root user with minimum privileges? Is it's password "blocked" and shell set to a dummy value such as /dev/null ? If the password is really needed, is it set to min 8 chars with numbers, letters and punctuation?
• If it runs as root, to bind to a low port, does it immediately fork to a non privileged user to minimise risks?
• Are file permissions set so that only the application user can read/write files are there are no world read/writeable files?
• Does the application write log permissions securely? Does it write passwords in the install log (don't laugh - it's common!).
• Does it chroot to a special directory? A good article on general chroot environments, also called "padded cells" can be read at Sunworld [2].

FTP server (ftpd)

• If you use Washington University wu-ftpd (for it's better logging, access control and other features), be wary. It has a history of bugs (e.g. see CERT advisories CA-93:06, CA-94:07, CA-95:16 and Auscert AA-97.03 and AA-1999.02). Use at least V2.6.0 or later.
• Configure /etc/ftpusers to list system accounts, so they cannot not be used for ftp e.g. disable use of ftp by root, add "root" to /etc/ftpusers.
  One way of doing this for all system accounts on your fresh new system is:
    awk -F: '{print $1}' /etc/passwd > /etc/ftpusers
• FTP can be selectively enabled per user via /etc/ftpusers or by using the following trick: For users who shouldn't have ftp access to this machine, give their accounts a non-standard shell (such as bash or tcsh) and don't enter this new shell into /etc/shells. FTP access will be denied. Conversely, if a non-standard shell is required, it must appear in /etc/shells for FTP to work correctly.
• Enable logging: adding the "-l" option in /etc/inetd.conf (the additional -d option adds debug output).
• FTP can be restricted on an IP address or hostname basis with the tcp wrappers.
• For anonymous ftp, be VERY careful. A chroot environment is a MUST. See the in.ftpd man page which includes a script. Avoid allowing uploads. If uploads are necessary, don't allow downloads of uploaded files, hide uploaded file names and don't allow them to be overwritten (otherwise you may end up as an illegal software repository).
• For normal "user ftp", chroot is also recommended.
• Put the ftp data area on a separate disk partition and mount it nosuid.

DNS servers

• Use the latest BIND (Berkeley Internet Name Server) rather than Sun's named. (Note that Solaris 8 includes the newer BIND 8.1.2). BIND has more features, is easier to debug (with dig) and is updated quickly when security problems are found. See [13].
• Use 8.2.2-P5 or later (which includes security fixes).
  Note that bind patches typically also include a shared  libresolv.so that must be installed even if you install your own version of BIND.
• Create a "dns" user and group and start with "/usr/local/sbin/named -u dns", so that BIND will fork and change identity to the dns user (rather than the all powerful root). Give the dns user read access to BIND configuration files (they should belong to root however).
• Restrict zone transfers to the IP address of secondaries (in /etc/named.conf).
• Consider installing BIND in a chrooted environment.
• Troubleshooting:
  • Use nslookup or dig to check server results.
  • Client: Check /etc/nsswitch.conf and /etc/resolv.conf if you're having DNS client side problems. Start nslookup with the "-d2" option to get buckets of debugging info. Try killing the nscd daemon.
  • Server: Use spaces and not tabs in Sun's old /etc/named.boot (bind uses named.conf). Start named with the debug option "-d" and read the console & logs. Typically logs are found in the syslog "daemon" section.
  • See also [13]
  • To get statistics from the name server, do the following
         kill -ABRT `cat /etc/named.pid`
    This puts the statistics into the file /usr/tmp/named.stats.
  • Send a HUP signal to named, to reread the config file after changes.
        kill -HUP `cat /etc/named.pid`

HTTP servers

• If content doesn't change often, store it on CD-ROM, it simplifies change control and provides greater resilience to the hackers favourite of "site defacing". The HTTPD server will cache pages, ensuring that access speed does not depend on CD-ROM speed.
• Separate "simple read-only" content servers from E-commerce servers.
• Check out the CIAC Bulletin J-042 which describes best practices for Internet web servers.
• Solaris 8 setup: Apache 1.3.9 is included in Solaris 8, installed in /usr/apache with configuration in /etc/apache. The daemon will start if /etc/apache/httpd.conf exists. The default configuration runs as the user nobody. Server root is /var/apache, containing htdocs, logs and cgi-bin.
• Solaris 7 Setup for a simple Apache server:
  • Download (e.g. from sunfreeware.com) and install as root in /usr/local/apache. Create "apache" user & group with blocked password, or just use the nobody group.
  • httpd.conf changes: Set ServerName, set User & Group=apache.
  • File permissions -  by default everything will belong to bin, which is fine. Remove world access, give minimum rights to apache group:
    chgrp -R apache /usr/local/apache;  chmod -R o-rwx /usr/local/apache;
• Tweaking:
  set ServerAdmin to the webmaster email,
  disable UserDir (unless User's really are allowed host pages form their HOME),
  ServerSignature could be set to no to avoid informing users of the Apache version, but it may be more useful to set it to email, to allow users sent bug reports to the webmaster.
  Disable the /manual/ alias, unless you really want to publish the Apache documentation.
  Production environments will probably not want server-info, server-status or perl-status enabled (they are disabled by default).
  Make sure that access to the root file system is disabled:
  <Directory />
  Options None
  AllowOverride None
  </Directory>
  Don't allow viewing of .htaccess files:
  <Files .htaccess>
  order allow,deny
  deny from all
  </Files>

Email gateways (smtp)

Consider using SMAP on email gateways exposed to the Internet. If using sendmail, keep up to date with fixes, or consider using Postfix. See also [16].

10. Going live

Preparing to go live

1. You probably won't need CR-ROMs or floppies anymore, so disable the volume manager in /etc/yassp.conf (if it was still enabled, which it not by default).
   If you do need to mount a CD in the future, start vold manually & check for new devices:
       drvconfig; disks; vold &; volcheck; df -k
2. Consider installing a script to check that important daemons are running. Install monitor_processes.pl  [3] and add a root cron entry:
   ## Check that important processes are running during office hours:
   ## [If you run 7x24, modify accordingly]
   0,30 8-19 * * 1-5 /secure/monitor_processes.pl sshd httpd
3. If partitions such as /opt or /usr had to be mounted read-write during the application install/testing, mount them read-only now.
4. Reinitialise tripwire (or equivalent integrity checker).
5. Backup the system to two tapes, one offsite.
6. Run a network scan on the system, to ensure that only expected services are visible. A commercial tool such as ISS or a free one like Nessus, nmap or Satan should do the job. Print out the results and archive.
7. If possible, have additional people do the final testing, just in case something was forgotten.
8. Test in detail - What works? What is forbidden? Check console/log entries. Does the system behave as expected? Watch the logs very frequently during the first few days of production.

Going Live

Connect to the live network. Test in detail. Check log entries. Does the system behave as expected?

Have applications been tested in detail, by different people with different points of view, from different access points on the network?

Regular maintenance

The following activities should take place hourly, daily, weekly or monthly, depending on how critical the system is:

• Check the status of patches with Sun's Patchdiag, update as needed. Be very wary of kernel patches (test on a non-production machine).
• Check all logs for errors and unusual activity: syslog (/var/adm/messages or /var/log/*log, depending on syslog.conf), /var/cron/log, last, /var/adm/sulog, /var/adm/loginlog, application/server logs.
• Write scripts to report if critical daemons die, or if important systems cannot be pinged.
• Run tripwire (or equivalent integrity checker).
• Be regularly informed of new vulnerabilities and security issues, either by subscribing directly to CERT, CIAC [18] and the vendor security lists (Sun, Microsoft, etc.) and/or, subscribing to newsletters such as  SecurityPortal's weekly or the SANS weekly/monthly [19].

Yassp Overview

Jean Chouanard's Yet Another Solaris Security Package [1] is a set of scripts used internally by Xerox that are being offered to the world for free. Jean is trying to make the scripts more general, to make them more useful outside of Xerox. The first version appeared in summer 1999 and adheres pretty closely to the SANS Solaris Guide. We now have an improved yassp beta 5 (May.2000) that allows more individual tuning and can be used for bastion hosts, servers and workstations. It's been tested by many admins, has received much input from experts and tried to pull together all known issues of Solaris hardening into one bundle.
An example of yassp output is useful in understanding what it does: yassp_output.txt.

• Improves file permissions according to Casper Dik's "fix-modes" package [8].
• Cleans the Solaris package database, so that it is consistent with installed files.
• Tunes Solaris according to Jens-S. Vöckler's "nettune" [10], i.e. kernel and network parameters for better performance & resistance to Denial of service attacks and buffer overflow attacks. Jean also added some ideas from the Titan Project  [11].
• Hardens the system for typical bastion host usage. The hardening has evolved from the original SANS paper and includes improvements from many other hardening scripts and documents.
  The system administrator can selectively allow services by editing the (easy) /etc/rc.conf and /etc/yassp.conf.

There is also a tarball available in addition to the packages. It's only recommended for experts. This tarball includes the hardening packages plus "GNUgzip GNUrcs SSHsdi WVtcpd PARCdaily". It doesn't allow any more flexibility, just bundles a few binary packages, it is only for SPARC Solaris 2.6/7 and not as well tested. The tarball does not ask you what you do or don't what to install/harden. It installs SSH1 with Jean's patches for SecurID (which are very useful). The GNU stuff is installed in /opt/local.

YASSP Goodies

• Free.
• Documentation:Html doc available on line that is pretty good.
• Comprehensive, works, tested on bastion hosts and on a Solaris 2.6 workstation with CDE.
• Can be backed out (great for testing).
• Package format (if you like packages)..
• Includes Kasper's fix-modes and Jens's tuning scripts.
• Supports supports Solaris 2.[567], and also the i386 arch (not the tarball). 80% works on Solaris8.
• /etc/rc.conf and /etc/yassp.conf allow sysadmins to configure what services they do and don't want via these simple files (which look suspiciously like BSD!).

Yassp disadvantages

• Doc could be improved
• The "backout" works, but you lose any changes you made to some system files, since they are replaced rather than edited.... unless you check out /tmp/SECCLEAN before you next reboot. (It is planned to change this to /var/tmp/SECCLEAN in a future version).
• Deinstallation/backout leaves traces (beta#5): /etc/init.d/rpc still has yassp entries and old inetinit/inetsvc are left lying around.
• "/opt/local": I believe most people use /usr/local, like myself.....
  • The path in /etc/default/su assumes you use /opt/local/{bin,etc,sbin}.
  • Yassp installs some scripts in /opt/local/sbin, which means that /opt/local will be created for those of us who don't yet use it.
  • Proposed fix: On installation, if /usr/local exists, but not /opt/local, then create a link from /opt/local to /usr/local. This will ensure that paths are correct and the yassp tools are installed in the same tree as other tools.
• Some things cannot be easily customised on the install scripts (like PATH above).
• Solaris 8: Although yassp does not officially support Solaris 8, it installed and worked pretty well and is useful.
  1. Yassp did not have replacements for /etc/init.d/inetsvc and /etc/init.d/inetinit and so "partially failed" the install. These files need to be put back manually.
  2. Solaris 8 already has an /etc/ftpusers, so installing the yassp package gave a warning about replacing this file.
  3. One of the tune up parameters failed, disabling the following line in /etc/init.d/nettune solves the problem:
     ndd -set /dev/ip ip_ire_flush_interval 120000
  4. A file permissions cleanup script for 2.8 is not yet available (although fix-modes seems to support it).
  5. "/usr/sadm/lib/wbem/cimomboot start" is running: Disable it.

Yassp - suggested improvements

• Add Solaris 8 support
• Change the Root description from "Super-User"  to "root MACHINENAME" in /etc/passwd
• Consider adding chroot for DNS, anon ftp, user ftp, apache, and chroot support for syslog...
• Automatically check patches.
• Fix /opt/local and deinstallation problems as noted above.
• Locked accounts: Use /usr/local/bin/noshell, which logs attempts to use these accounts to syslog. Jean uses /dev/null, since it works on both x86 and SPARC platform (otherwise one needs a binary for each). I still think it's a better idea to use noshell.
• Why is both yassp.conf and rc.conf needed? (A merge in planned in the next release).

Additional Notes

This article has been very specific, in the interest of making it practical. However, each security administrator has his own methods and each site has different requirements.

• For increased security: don't just remove startup files, but also program binaries.
• use Jumpstart: for installing large numbers of hosts, or to be able to quickly produce a new hardened host (perhaps after a failure). Jumpstart is not trivial to setup, but worthwhile when installations are frequent.
• On high availability servers where the console is exposed:
  • The Stop-A keyboard sequence can be disabled by setting KEYBOARD_ABORT to disable in /etc/default/kbd.  Of couse "STOP-A; sync" won't work ony more which can be a major inconvenience, I suggest only doing this in physically insecure environments.
  • Switch on EEPROM security (requires a password to be entered each time the prom prompt is accessed):
    eeprom security-mode=command
    This shoulod also be used with care, as forgetting the password can render a machine useless.
• Versioning: Use RCS or CVS to manage versions of configuration files. This is especially important when several administrators manage a host, changes are frequent, etc. Versioning should also be used with Jumpstart scripts & config files. An alternative is to make a copy of each config file BEFORE making changes. Append a date to the file, e.g. /etc/syslog.conf.19991023.
  (Yassp's PARCdaily does this automatically, but I've not tested it).
• Time Synchronisation: If you think you might have to produce evidence in court, use NTP to synchronise time, not rdate. Prosecutions have failed because, on busy servers, a few seconds difference could be used to insist that there is a doubt about the identity of a session. (This happened to a bank in Australia trying to prosecute an Internal Attacker).
  If using NTP, either get a (cheap) radio clock (if you are near an appropriate transmitter - e.g. the Stuttgart transmitter in Germany), or setup a dedicated bastion host as a 2nd stratum NTP server, which uses 3 1st stratum sources. Setup the firewall filter to allow only NTP from the bastion to the three 1st stratums and allow Intranet hosts to query the bastion.
  NTP can also be setup for higher security by configuring DES+MD5 authentication keys on server and client, see man xntpd.
• Logging:
  • in addition to centralised syslogs, you might like to keep an additional local copy - in case the syslog server goes down or is subjected to a denial of service attack. Make sure /var is a separate filesystem if you use local logging, to avoid root being filled and the system stopping.
  • There are several improved syslog daemons [17] .
• Intrusion detection:
  • Regular logfile analysis can be implemented with customer scripts or tools such as logcheck and swatch [6].
  • Integrity checkers: see section above.
  • Doug Hughes has written tocsin and klaxon [4], two free tools that listen for network scans and connections to particular ports. Tocsin only needs to be installed once per subnet and klaxon is installed in inetd.conf for key services that should be closed and where you don't expect a connection (but would like to know). I've test klaxon on Solaris 2.5,2.6 and 2.7. Tocsin works on 2.5 and 2.6, but won't work on 2.7 (compiled on 2.5 or 2.6). A sample startup script for tocsin is S11tocsin [3].
  • I have written a perl script monitor_socket.pl [3] that listens to a list of sockets and notifies by email and syslog if a connection is received. Like klaxon above, but does not run from inetd. Originally written to detect Sybase and Satan connection attempts.
• RPC is a major security problem due it's lack of authentication, encryption and it's use of dynamic ports. Normally RPC should not be run on bastion or other sensitive hosts. However, come programs insist on having RPC e.g. CDE, OpenWindows, Disksuite and Legato Networker. 
  • If at all possible, avoid RPC (hence I prefer manual mirroring with a cpio script for warm standbys, rather than Disksuite). If RPC really is necessary, then....
  • If you are using Solaris 8, the bundled (free) Sunscreen Firewall can be use to filter who can access the rpc services (since the Sunscreen includes an RPC filter engine), or...
  • Install Wietse Venema's RPCBIND [14] which includes tcp wrapper like access control for RPC connections. Summary of features (extract from the README):
    - host access control on IP addresses. The local host is considered authorized. Host access control requires the libwrap.a library that comes with recent tcp wrapper implementations.
    - requests that are forwarded by the rpcbind process will be forwarded through an unprivileged port.
    - the rpcbind process refuses to forward requests to rpc daemons that do (or should) verify the origin of the request: at present, the list includes most of the calls to the NFS mountd/nfsd daemons and the NIS daemons.
    - the rpcbind process refuses REMOTE requests sent to high-numbered UDP ports (instead of TCP or UDP ports 111). High-numbered ports are opened by the rpcbind server as a side effect of other activity. These ports could be abused to bypass packet filtering restrictions. See the advisory (and addendum) on http://www.secnet.com/
    It is based on the Solaris 2.3 sources for rpcbind, which were made publicly available by Sun in 1994. I have tested rpcbind V2.1 (April 1998)  on a Solaris 2.6 workstation running CDE, without any problems.
    Verdict: Recommended, however one wonders whether Sun have not made any impovements to rpcbind since 1994. It would be useful if Sun incorporated Venema's changes. Solaris 8 is available with sources, so perhaps Venema can produce a new version based on these new sources later this year?
    By default, logging is to syslog MAIL.INFO, which is a little strange, ==> change it to AUTH.INFO in the Makefile before compiling.

References

Other useful links:

• Security Portal Research Centre
• Free Tools
  Top, gzip, lsof, traceroute, perl: www.sunfreeware.com
  Rdist www.magnicomp.com/rdist/rdist.shtml
  Network weaknesses scanners: Nessus, Satan, Nmap.
  Intrusion Detection: snort
• Firewall Feature Comparison www.spirit.com/cgi-bin/report.pl
• Education: A good reference for any security issue is the SANS Security Roadmap.
  Visiting the SANS security conferences is worthwhile, they have a workshop on Hardening Solaris, among  other interesting courses.

Changes to this article

25.Oct.'99  First Publication.
08.Nov.'99 Thanks to the following people for providing feedback and suggestions after the initial publication: Glenn Brunette, Andrew van der Stock, Darrel Goeddel and Kamal Kantawala.
28.Jan.'00 Integrate first Yassp version
08.Mar.'00 Rearrange links and references.
21.Apr.'00 Rewrite. Adapt for yassp beta3, Solaris 8, Sunscreen, Apache. New: /etc/weekly script.
13.May.'00 Update: Links, Venema's tools, Yassp beta5, tripwire, feedback Jean Chouanard. 
                New: RPC, smtp. (Second Publication)
16.may'00 Update: yassp#5 deinstall problems, rpc.
23.may'00 Update: tripwire URL and check parameters. Cert references. Fix Links.
20.jun'00   Update: typos in OS section, More Solaris patches, Disk mounting (vfstab options).

Seán Boran is an IT security consultant based in Switzerland and the author of the online IT Security Cookbook.