Solaris Security Digest Tips: May-Dec. 2000

By Seán Boran (sean at boran.com)



December 20, 2000 - The Solaris Security Digest came to life in May 2000, and each week we have tried to include a useful tip. This document collects those tips in FAQ style. Just browse the table of contents to find items that interest you.

Thanks for reading the Digest this year. Enjoy the holidays and look forward to 2001. Feedback and comments are welcome as usual.



Contents

  1. Sun documentation
  2. Cold mirroring with mirror_boot.sh
  3. The "ok" prompt
  4. Version control of configuration files with SAVEIT
  5. ntop
  6. TCT: The Coroner's Toolkit (TCT)
  7. Sara and Tara
  8. SunShield BSM/C2
  9. Disk mounting (vfstab options)
  10. Host Auditing: audit1.sh audit2.pl
  11. tocsin
  12. Using TSIGs in BIND
  13. "useradd" and "defadduser"
  14. Securing Sendmail for typical workstation usage
  15. IPFilter
  16. Tuning Syslog
  17. Limiting SUID Files
  18. Improving Disksuite Security
  19. Resources for Hardening Solaris
  20. C2, TFTPD
  21. Yassp beta#13
  22. Improving "usermod"
  23. Reconfiguring devices: boot -r
  24. Solaris on Headless PCs
  25. The dreaded patches: Patch management strategies & tools



Sun Documentation

Sun now publishes all user manuals, administration manuals, reference guides, etc., on docs.sun.com in several languages. Likewise, iPlanet documents are available on docs.iPlanet.com.



Cold Mirroring with mirror_boot.sh

Minimum downtime and prevention of data loss is important for most servers. The traditional solution is to use a RAID box to cover for disk failures. However, if the root /usr /var filesystems are on RAID and the raid controller goes, you have a problem (unless you have a fully redundant raid). I've been caught out twice over the last years by failed fiber channels and controllers, and I now use RAID for data disks, put system files on a normal disk, and mirror to an identical disk each night. If the boot disk dies, boot from the second disk.

The script mirror_boot.sh has been developed to do just that. Sample out is in mirror_output.txt. Read carefully the comments at the top of the script before using it!

How it works: Each night the offline disk is mounted and synchronized with the primary disk. The script is called from the root cron nightly. It mounts the spare disk under /newroot, copies all filesystems, installs a boot block and copies over a new vfstab. This creates a fully updated bootable spare disk. The results of the script are sent to the administrator via email.

Advantages: This solution is very simple (no complex hardware or software) and easy to understand in a crisis (when RAID complexity can be hell).

Disadvantages: Hot failover is not available, and a downtime of say 15 minutes has to be acceptable.



The "ok" Prompt

The "ok" prompt has a few hidden treasures. Bootprom is a very useful feature that gives Sun hardware an advantage above the rest: the possibility to remotely (via the serial line console) diagnose hardware problems, boot from disk/cd/network, or remotely install Operating Systems.

I've been using commands like go, resume, boot, reset, sync, printenv, setenv, probe-scsi, probe-scsi-all, test-all, etc., for many years, but had missed some obvious ones that are very useful, such as the following.
Note that "^" means "hold Ctrl key."

^P: previous command ^A/E: jump to line start/end
^N: next command escB/F: jump to previous/next word
^L: list lines ^K: delete to end of line
^R: repeat line ^W/D: delete previous/next word
show-devs [device-path]
Display all the devices directly under the specified device in the device tree; without device-path it shows the entire device tree.
disk-info: List all disks such that their path can be copied and used in the nvalias command (saves on lots of typing)

Warning: some commands like watch-net and test-net have caused systems to hang (for me); a cold restart was needed.

See also Sun's OpenBoot Collection documentation and its search feature.



Version Control of configuration files with SAVEIT

It's a good idea to keep previous versions of configuration files, to allow rollback, have an audit trail and coordinate between several administrators. Some sysadmins use rcs, some copy the configuration file to file.DATE beforehand, some do nothing :).

Francisco Mancardi from U&R Consultores [fman@uyr.com.ar] has written a simple script that can be used to save a copy of a file or directory before doing modifications.

Saveit is a little tool to make a backup of config files before you change them. It saves a copy under /Backup.d/DATE/ and logs "who saved what file" in /Backup.d/log-DATE. The existing directory structure is preserved under the backup directory.
It's a simple, but useful version control tool for text files. E.g.,

etc/[9]% saveit vfstab
copying file vfstab ===> /Backup.d/20000707/etc/vfstab

etc/[61]% saveit vfstab
copying file vfstab ===> /Backup.d/20000707/etc/vfstab.13:37

etc/[10]% ls -l /Backup.d/20000707/etc/vfs*
-rw-r--r-- 1 root sys 386 Mar 14 08:31 /Backup.d/20000707/etc/vfstab
-rw-r--r-- 1 root sys 386 Mar 14 08:31 /Backup.d/20000707/etc/vfstab.13:37


etc/[11]% tail /Backup.d/log-20000707
Backup base directory /Backup.d
Backup requested by root
Date (dd/mm/aaaa) 07-07-2000
Time 11:36
/etc/vfstab
--------------------------------------------------------------
Backup base directory /Backup.d
Backup requested by root
Date (dd/mm/aaaa) 07-07-2000
Time 13:37
/etc/vfstab

The advantages of this tool are: simplicity, and no clogging up the current directory with old versions of files, rcs directories, etc. It has been tested on Linux, Solaris 2.6/7 and OpenBSD 2.6.

Three versions of the script can be downloaded:



ntop

What's ntop?

ntop is a tool that shows the network usage, similar to what the popular top Unix command does [for processes]. ntop is based on libpcap and it has been written in a portable way in order to run on virtually every Unix platform and on Win32 as well. . .
ntop comes with two applications: the "classical" ntop that sports an embedded web server, and intop (interactive ntop) is basically a network shell based on the ntop engine.

If you're wondering who's using what bandwidth on a server or local network, ntop is a useful tool that is simple to install and has a pretty good GUI. It was first presented to the public in spring, and several papers have been written about it, for example Improving Network Security Using Ntop which discusses using it as an IDS sniffer.

A binary package of an older Solaris7 version is available on sunfreeware.com, the latest version's source doesn't (yet) compile for me on Solaris8. The Solaris7 package does run, however, on Solaris8, but doesn't have the interactive interface.

Ntop runs as root, and listens for http connections on port 3000. On connecting with the browser, several statistic tables, graphics, etc., show the network usage. It's not advised to run it on sensitive systems, until the security and availability issues have been fully understood.

I hope to get the newest ntop tested over the next few weeks. Results will be posted here.



TCT: The Coroner's Toolkit (TCT)

Once your system has been penetrated by an attacker, what do you (apart from panic!)?

The Coroner's Toolkit (TCT) by Dan Farmer and Wietse Venema can help with the analysis of the system in an effort to find out what happened, how and when.

Tips

Preparation



SARA and TARA

Advanced Research Corporation is providing free updates to Satan and Tiger, two old but useful auditing tools.

The Security Auditor's Research Assistant (SARA) is an updated version of the venerable SATAN network auditing tool.

Satan was a useful tool, with a good user interface, but it was not updated except for the Saint derivative. Bob Todd, the SAINT author, has been working on SARA since 1999. Some useful features in SARA include integrated support for nmap, samba and DDoS tools. A report writer for pretty summary reports is also available.

Tests show that it works on Solaris 8 and OpenBSD.

Problems:

TARA is an update of the 1994 "Tiger scripts" for host-based security auditing.

Since 'tiger' has not been updated since 1994, there were numerous changes made to the 'systems' directories. Output was streamlined to provide a more readable report file. Also, minor bugs in the 'scripts' directory were corrected. TARA was tested under Red Hat Version 5.x, 6.x, SGI IRIX 5.3, 6.x, and SunOS 5.x.

This is also a useful update. It has a "-H" option to produce HTML output. It's worth taking time to customize tigerrc to your needs before running it.

Problem: no signature DBs are available for newer Solaris versions.



SunShield BSM/C2

Francisco Mancardi from U&R Consultores [fman@uyr.com.ar], who gave us the "saveit" script five weeks back, has scripts to make it easier to read the Sun's C2 (SunShield BSM) auditing logs. The scripts and files described here can be downloaded. First we'll briefly run though setting up C2 (SunShield BSM) audit logs.

What is SunShield BSM?

Sun delivers a "C2" level auditing system for both SunOS (Sunshield) and Solaris (Sunshield BSM). It is bundled with Solaris 2. BSM allows the actions of specific users to be recorded and written to an audit file. However, the auditing is at the system-call level, meaning huge logs may be generated by simple user actions. Performance is also affected. The standard analysis tools praudit and auditreduce offer no high-level analysis of audit trails. Applications may also write to the audit trail.
Reference documentation: "SunShield Basic Security Module Guide" (Standard Solaris 2.x documentation). Man pages: audit(1m), audit_startup(1m), audit_warn(1m), auditconfig(1m), auditreduce(1m), bsmconv(1m).

Enabling and configuring SunShield BSM:

Install the audit2info scripts:

Try out audit2info:

First, produce an ASCII file on C2 auditing activity :
cd /opt/audit;
auditreduce | praudit > audit-data.txt

We can now analyze this output in different ways using the "audit2info" script. Links are provided to sample results.

Let's make the output from auditreduce | praudit above more readable:
./audit2info audit-data.txt > outfile

Show only activity concerning one username:
./audit2info -u bill audit-data.txt > outfile

Show only login/logout activity:
./audit2info -l ok audit-data.txt > outfile

Show only login failure activity:
./audit2info -l ko audit-data.txt > outfile

Show the complete output from auditreduce |praudit, but filtering out records that have fields with the string listed in the filter file default "filtro_audit"
./audit2info -f default audit-data.txt > outfile

The above example can be mixed as desired. . .

Notes using SunShield BSM:

More BSM articles have appeared since the above was published:

Implementing C2 Auditing in the Solaris Environment, by Kevin Wenchel and Stephen Michaels
http://www.sysadminmag.com/supplement/913c2.shtml



Disk Mounting (vfstab Options)

Several options can be set to improve the security and robustness of filesystems when they are mounted. Run the mount command to check that filesystems options are effective.

Mount option OS Description When to use it
nosuid 2.x Disables SUID programs, but also disables devices! /var or /home or data disks where no SUID programs, or devices (and hence chroot environments are used).
/tmp won't work either, unless it is on disk.
logging 2.7 or later keeps a transaction log within the mounted partition. The advantage is an almost instantaneous filesystem check - which may take a considerable while with larger harddisks, e.g., 18 GB. The disadvantage is the additional time spent writing the transaction log. /usr /opt /home

Recommended for all file systems except: root (if Veritas VxVM is used), or where lots of file accesses are expected.
noatime 2.7 or later allows mounting file systems without updating inodes at each access to any file. This will significantly speed up services like Web caches or news servers, which do a lot of file IO with small files. /var or any partition where lots of file access is expected (Web cache or news partitions).
size=100m 2.5.1 or later Allow /tmp to only use 100MB of swap space. The value could be set to say 30% of swap space. /tmp
ro 2.x Read-only
Mounting filesystems read-only provides only a limited protection against Trojans/attackers (if they get root, they can remount read-write), but it saves time fscking when shutting down, can improve performance (access times don't need to be updated) and can prevent the sysadmin from making mistakes or help detecting mistakes (accidentally deleting files, etc.). 
Mounting read-only is a major argument for maintaining separate file systems for /usr or /opt.

Note that to mount /usr read-only, /usr/local often needs to be on a different partition.

Be very careful when editing vfstab, e.g., an error on the / or /usr lines can render the system unbootable! (If this happens, boot from CD-ROM in single user mode, mount the problem disk, correct vfstab and reboot.) Some examples of vfstab entries are:

A simple server with only a root and /var partition, running Solaris 2.8:

fd                -                  /dev/fd fd - no -
/proc             -                  /proc proc - no -
/dev/dsk/c0t3d0s1 -                  -    swap  - no  logging
/dev/dsk/c0t3d0s0 /dev/rdsk/c0t3d0s0 /    ufs   1 no  logging
/dev/dsk/c0t3d0s7 /dev/rdsk/c0t3d0s7 /var ufs   1 no  logging,nosuid,noatime
swap              -                  /tmp tmpfs - yes size=100m

and on a larger server:

fd                -                  /dev/fd fd - no -
/proc             -                  /proc proc - no -
swap              -                  /tmp tmpfs - yes size=200M 
/dev/dsk/c0t8d0s0 /dev/rdsk/c0t8d0s0 /    ufs   1 no  logging
/dev/dsk/c0t8d0s1 -                  -    swap  - no  -
/dev/dsk/c0t8d0s4 /dev/rdsk/c0t8d0s4 /usr ufs   1 no  logging
/dev/dsk/c0t8d0s6 /dev/rdsk/c0t8d0s6 /var ufs   1 no  nosuid,noatime,logging
/dev/dsk/c0t8d0s5 /dev/rdsk/c0t8d0s5 /opt ufs   2 yes logging



Host Auditing: audit1.sh audit2.pl

Auditing the security of an existing Solaris system can be time-consuming, and often requires on-site visits. There are several commercial tools and a few free ones (e.g., Titan and the Coroner's Toolkit) that help, but they can be complicated and require local installation. A few weeks back we presented a small Bourne shell script for simple auditing. This has now been updated and complemented with a second script.

Purpose:
For situations where a "quick audit" of the system is required.
simple to run; the auditor could even give it to the sysadmin, ask him to run it at night and send the results back to the auditor (perhaps by encrypted email).
Its simplicity makes it easy to verify; it is not as thorough as other tools, but it is small and easy to understand.
This script automates the gathering of the information only; of course, the difficult part is the interpretation and deciding what countermeasures to take.

This has now developed into two scripts:

  1. audit1.sh (Bourne shell): This script is designed to run quickly and gather as much information as possible about the system. No file searches are conducted, to keep it fast.
    Tested on: Solaris 2.6/7/8, OpenBSD 2.6, RH Linux 2.2.12-42 (Solaris is best supported)
  2. audit2.pl (perl): This second script searches the entire filesystem, listing SUID, SGID, world-writeable, group-writeable files. It also lists trust files and their contents. Finally it lists files with weird names (e.g., containing punctuation characters), which might be danger or a sign of penetration. On a large server with 100GB disks, this can take a few hours to run.
    Tested on SunOS 5.5/6/7/8, OpenBSD 2.6.

Please read the headers in the scripts before using.



tocsin

Doug Hughes (Doug.Hughes@Eng.Auburn.edu) wrote tocsin, a "featherweight network intrusion detection system" back in 1996. It is a free tool that listens for network scans. Tocsin only needs to be installed once per subnet, unless switches are used, i.e., all nodes do not see all traffic. It uses DLPI [see note 1] kernel-level packet filtering and runs out of the box on SunOS and Solaris to catch port and stealth scans (SYN, FIN, ACK, Xmas, RST, etc.). Alert messages are logged to syslog "auth" startup and shutdown messages are logged to "daemon."

Over the last few weeks tocsin has been significantly improved and version 2.1.1.2 released.

fixed log file permissions
new tcp or udp options (:t or :u) per service
ported to Solaris 8 sparc/Intel
new common package for Solaris sparc/Intel
change UID to nobody after binding to the network
new man page and startup file "S70tocsin" added
New options: -T tcp only, -D destination network only, -O log IP options, -I invert port matching filter conditions.

This amazing little tool (it's only 20kB) is simple, but works quite well. The new options allow significant reduction of false positives. For this article, it was tested on Solaris 2.7-2.8/sparc and Solaris 2.8/Intel, but it should run on Solaris 2.6 and possibly 2.5.1 also.
tocsin can be downloaded in source form (tocsin.tar.gz), or a Solaris package (AUBtocsin), from:
ftp://ftp.eng.auburn.edu/pub/doug/

Doug has done great work with tocsin. If you've been waiting for an simple but effective IDS, this is worth a look.

Note 1: DLPI is the Datalink Provider Interface (a standard based on ISO 8886 and 8802 for Streams-based kernel implementations of packet filtering)



Using TSIGs in BIND

The DNS server, BIND, provides some advanced security features in its latest release. Here we examine one: the use of TSIGs (transaction signatures) to authenticate zone transfers. Zone transfers are usually limited to a list of IP addresses (via the ACL mechanism) which correspond to official DNS servers for a zone. Since only IP addresses are used, this mechanism is open to spoofing. BIND 8.2 and later allow authentication and verification of zone data. A key is configured on primary and secondary name servers, and the servers are configured to use this key to sign messages exchanged between the servers. It's important that the server times are synchronized. If the transfer is not authenticated with the correct key, no zone transfer may take place.

Let's look at an example where we use TSIG to restrict the zone transfers between a DNS primary "prim" (IP address 10.1.1.2) and DNS slave secondary "sec1" (IP address 10.1.2.2).

a) Generate an MD5 key, which will be used as a shared secret between the DNS servers. The "dnskeygen" tool is used; the key is written to a file.

# /usr/local/sbin/dnskeygen -H 128 -h -n prim-sec1. 
Generating 128 bit HMAC-MD5 Key for prim-sec1. 
Generated 128 bit Key for prim-sec1. id=0 alg=157 flags=513
# cat Kprim-sec1.+157+00000.private
Private-key-format: v1.2
Algorithm: 157 (HMAC)
Key: bFs2bXnLTYTI7r0WJv7HMA==

b) Create an identical key entry on both servers in named.conf:

key prim-sec1 {
	algorithm hmac-md5;
	secret "bFs2bXnLTYTI7r0WJv7HMA==";
};

c) Add an ACL on both servers to limit transfers to specific hosts, e.g.,

acl "my-nameservers" {
	localhost;
	10.1.1.2;
	10.1.2.2;
};

d) For each host in the ACL, tell the BIND which key to use (do this for each server), for example on the primary:

server 10.1.2.2 {
	transfer-format many-answers;
	keys { prim-sec1 ; };
};
zone "mytestdomain.com" {
          type master;
          file "mytestdomain.hosts";
          allow-query    { any; };
          allow-update   { none; };
          allow-transfer { my-nameservers; };
}

d) Restart both named servers (send a HUP signal), then check the (syslog) logs for errors.

Testing if it works:

So it's really not so difficult to significantly increase the security of your zone transfers. It is important that the file permissions on named.conf be restrictive so that it cannot be read by everyone on a system; the secret string used in the key must remain secret.



"useradd" and "defadduser"

The useradd tool is a typical way of adding new accounts to the system. It adds a new user to the /etc/passwd and /etc/shadow (and on Solaris 8 /etc/user_attr) files. For example:

useradd -c "Seán Boran" -f 60 -g ftpusers -m boran

This creates the new user "boran" belonging to group "ftpusers," an appropriate home directory (with copies of skeleton files from /etc/skel) and sets the inactive timeout to 60 days.

Apart from adding users, system defaults can also be set (with the "useradd -D" option), so that it is easier to use. For example:

useradd -D -b /home -f 60 -g ftpusers

Here the default parent directory is /home, accounts are inactive if idle for 60 days, the default group is "ftpusers."

The first time "useradd -D" is run, a file /usr/sadm/defadduser is created, which contains the list of defaults. This file can be edited manually. For example, on a default Solaris 7 it contains:

defgroup=1
defgname=other
defparent=/home
defskel=/etc/skel
defshell=/bin/sh
definact=0
defexpire=

One would expect that the default shell could be changed with a command like the following, but this does not work on Solaris 7 or 8.

useradd -D -s /usr/local/bin/bash

So the only way is to edit the /usr/sadm/defadduser file manually, for example:

defshell=/usr/local/bin/bash

Solaris 8 contains a few additional defaults in /usr/sadm/defadduser for the new Role Based Authentication Control (RBAC):

defauthorization=
defprofile=
defrole=

If /usr/sadm/defadduser is copied from a Solaris 8 machine to a Solaris 7, the additional Solaris 8 settings are silently ignored. This is very useful, as one could synchronize this file across an array of Solaris servers without worrying about versions.

See also: useradd (1m), userdel (1m), usermod (1m)



Securing Sendmail for Typical Workstation Usage

On Unix and Linux, sendmail is invariably installed as an email server, running as a daemon. Many users/sysadmin think sendmail needs to run as a daemon on email clients and servers alike.

The sendmail daemon does not need to run on an 'Email Client'.

This erroneous installation behavior seems to be a default that goes back 15 years. It is difficult to understand why vendors have not corrected this. Few email servers are needed in an organization. The default should be to only send email, since most hosts are workstations or servers that do not need an SMTP server. Running sendmail as a daemon increases security risks:

So how is an email client set up? Assuming we want to deliver all email from our workstation to one SMTP server, the steps are:

1. Define the mail server: add an entry with the email server name and IP address to /etc/hosts. Then add an alias "mailhost" for this machine. There should also be an entry for the workstation, and workstation.DOMAINNAME.

2. Configure sendmail.cf (in /etc or /etc/mail), so that all email is sent via "mailhost":

DSmailhost
DRmailhost
DHmailhost
O FallbackMXhost=mailhost

3. Stop the sendmail daemon from starting automatically:

mv /etc/rc2.d/S88sendmail /etc/rc2.d/.no_S88sendmail

and kill the current daemon.

4. Add appropriate aliases to /etc/mail/aliases, for example:

root: Your.Name@Yourcompany.com

5. If email cannot be delivered immediately (due to server congestion for example), add the following entry to the root crontab to check for and send queued email every hour:

## process the email Q
0 * * * 1-5 /usr/lib/sendmail -q

6. Finally, test that email is being correctly delivered, by sending an empty test email to "root" (or some other local alias) and to your usual email address. The "-v" option is added to mailx so that we see the complete dialog with the email server, to convince us that it really is working correctly.

mailx -v -s test_email root </dev/null
mailx -v -s test_email John.Doe@YourCompany.com </dev/null

Troubleshooting: look carefully at the output of the mailx commands above, and any error emails received in the root mailbox. Check that a "ping" to mailhost works as expected. The SMTP server may be configured only to accept emails from known workstations or workstations with a particular domain (in which case /etc/resolv.conf - DNS must be set up correctly and the workstation must use the same hostname as is listed in DNS).

Hopefully Sun and other vendors will wake up and stop the sendmail daemon by default in future OS releases. It would certainly help reduce the number of unwitting open email relays on the Internet.

Finally, here is a list of references for the SMTP server administrator:

Securing Sendmail sendmail.net/?feed=000705securitygeneral
SMAP & FWTK (Firewall Toolkit) www.fwtk.org
Sendmail www.sendmail.org 
Postfix www.Postfix.org 
Anti-Virus Mail Scanner amavis.org
Scan4Virus - Virus Scan Wrapper for Qmail www.geocities.com/jhaar/scan4virus
ORBS list of open relays www.orbs.org 
MIME Defanger www.roaringpenguin.com/mimedefang



IPFilter

I had always wanted to try out Darren Reed's IPFilter firewall, and came across a great book on the topic, which I read cover to cover. It also covers ipchains. Although the book's focus is on using IPFilter on OpenBSD, it's equally valid for Solaris.

The book was detailed enough to see what IPFilter configurations look like in the real world. That's where I ran across two problems that seem like showstoppers to me, since I wanted to use IPFilter in a large, complex firewall:

So my summary of IPF is:

Features: IPFilter offers filtering of protocol (UDP or TCP), IP fragments, ports (and ranges), IP options, TCP flags, ICMP type/code and provides NAT, logging, transparent routing, VLSM (Variable Length Subnet Masks). In addition, redirection of services "transparent proxy" and packet state can be analyzed to check that TCP packet ack/sequence numbers are correct.

Advantages: free, powerful, source code, works on many Unix variants, probably easier to use than ipchains.

Disadvantages: no GUI, requires expert configuration. No definition of address groups is possible, which could make the rules for a firewall protection of a large number of networks very complicated. Likewise, no definition of groups of protocols is available. The filter engine is not intelligent: it does not understand applications protocols like RPC, FTP which makes spoofing the "keep state" feature a risk.

If you're interested in IPF, some references:
•  A great book to buy is Building Linux and OpenBSD Firewalls, ISBN 0-471-35366-3 Sonnenreich/Yates/1999, and check out the companion Website www.openlysecure.org.
•  cheops.anu.edu.au/~avalon/ip-filter.html
•  ftp://coombs.anu.edu.au/pub/net/ip-filter
•  mailing list at majordomo@coombs.anu.edu.au with a subject "subscribe ipfilter".
•  IP Filter Based Firewalls HOWTO
•  SecurityFocus Introduction to IP Filter, Introduction to IP Filter Part 2.



Tuning Syslog

The logging service delivered with Unix, syslogd, while very useful, is often not understood, badly documented and under-utilized. Here we provide a few tips to allow you to use syslog better. Note that Sun's syslog does not quite work like other variants such as Linux (not as good/flexible).

The syslog daemon runs by default on Solaris and collects messages from system and application daemons. Having collected the messages, it then either writes them to a file, sends them to a user or to a centralized syslog host. You can send messages manually to syslog via the logger tool (see the logger man page).

The file which determines syslog's behavior is /etc/syslog.conf. The default configuration (see link below) from Sun is many years old. What does it do?


How can we improve on this?


So how do we change syslog's configuration?

The configuration file is a little quirky, and the man pages are not perfect, so first we look at the key elements.

Each line in syslog.conf is an instruction to send specific types of messages to a certain destination. The type of messages depends on the facility and priority.

The facility corresponds basically to the type/name of the application. There are 19 allowed values: kern, user, mail, daemon, auth, syslog, lpr, mark, news, uucp, cron, local0... local7. As you can see, many of the names correspond to typical Unix daemons. The kern facility corresponds to the Solaris kernel, and we find the boot messages there. The 8 facilities local0 to local7 have no special applications and can be attributed as desired. The facility used by most applications is the daemon facility. The mark facility is used to add regular timestamps.
Tip: An asterisk ("*") can be used to denote all facilities.

The message priority can be emerg, crit, err, warning, notice, info, debug or none. The least urgent/most verbose is debug, and conversely, emerg is reserved for very urgent messages.

The destination can be a filename (starts with a slash), a remote host (starts with an @), one or more usernames (separated by comma, or "*" to indicate all logged-in users).

So an entry like "auth.info   /var/mylog" would mean all authentication messages of priority "information" or higher should be written to the file /var/mylog.


Gotchas


Debugging tips

Start syslogd with "-d"; it will show a matrix of what messages will be sent to what destinations. Then use the logger tool to send message with different facilities and priorities.


Security

Syslog uses udp transport, so message delivery is not guaranteed to remote loghosts - although if dedicated hosts are used inside critical zone, it should pose little problem. You may wish to keep a local copy of certain logs, even if a central log server exists. It is not advised to have your syslog port open to the Internet. An attacker may decide to fill your logs with junk.

By default syslog sits on the network, listens for messages from anywhere (local or remote) and processes them. This is fine for a syslog server, but the typical workstation/server does not need to receive messages. In Solaris 8, syslogd can be started with the "-t" option, which indicates that it should not listen to messages from remote hosts.


So now you've been introduced to syslog, how about a few sample configurations?

Our starting point is Sun's default:
http://www.boran.com/security/sp/solaris/syslog_sun.conf

Custom config #1: This example is a configuration I've used for 7 years with many machines. It's also used in YASSP beta 9 to beta 11.
http://www.boran.com/security/sp/solaris/syslog.conf   
Function: If a loghost is defined, catch all messages and send to the log server - otherwise split messages to separate files per facility (I prefer not to have DNS messages mixed with SSH and POP messages), and copy all alert or higher messages into a separate file.

Disadvantages:

  1. Since /var/adm/message is replaced by many individual files, some users/administrators won't be able to find the logs.
  2. The "dmesg" command will not work, since it examines /var/adm/messages. Some people will rely on dmesg. I only found this out after using the configuration for 7 years!
  3. It's more complex and hence more difficult to understand.

Given the above disadvantages, I now tend to recommend this configuration for central log servers, or large servers with many different applications and expert administrators.

Custom #2: For small servers and workstations.
Use /var/adm/messages rather than separate files per facility.
http://www.boran.com/security/sp/solaris/syslog_2.conf   

All messages of priority "info" or higher are sent to the loghost or logged to /var/adm/messages. Minimal messages are sent to the console and users.

By uncommenting "section 5" in this example, a local log of "errors, authentication and system boot messages" will be kept, even if all messages are forwarded to a cental loghost.

We've covered syslog in quite a bit of detail and presented several examples that should help you get the best from your system logging. One more detail to note is that there are other implementations of syslog available. These offer either better security or improved functionality. I've not yet tested them, however, and can't make recommendations.

syslog-ng (tcp connections, content filtering, encryption, authentication)
http://www.balabit.hu/products/syslog-ng  

secure syslog (encryption and authentication of a trusted auditors machine)
http://www.core-sdi.com/english/slogging/ssyslog.html

Nsyslogd (tcp connections & SSL)
http://coombs.anu.edu.au/~avalon/nsyslog.html  



Limiting SUID Files

Files which have the SUID bit set (an "s" where the execute bit for the owner/group is shown in &quotls" listings) allow the user executing the program to assume the identity/group of the owner of the program. This is typically used to allow normal users to access certain functions typically only allowed to root, for example binding to low ports, mounting a floppy disk, etc. The problem is that historically, many security weakness have been found in such programs allowing attackers with local accounts to become root by exploiting buffer overflows, race conditions, etc.

Solaris has many "SUID root" binaries and each one presents a risk, so when hardening systems it is advisable to disable as many SUID programs as possible.

The purpose of this section is to provide a brief overview of the subject. A list of documents and scripts for disabling SUID files is provided.

What SUID files are on the system?

The find command can be used to list all SUID files:
find / -perm -u+s -ls

or all SGID files:
find / -perm -g+s -ls

Example listing of SUID/SGID files on a Solaris 7 system:
http://www.boran.com/security/sp/solaris/suid_sol7.txt

Example listing of SUID/SGID files on a Solaris 8 system:
http://www.boran.com/security/sp/solaris/suid_sol8.txt

How should we handle SUID files? Possible courses of action, in order of preference, are:

What SUID files need to be limited?

SUID references and further reading:

  1. Solaris 7 Setuid/Setgid Files Information Systems and Technology, University of Waterloo
    (Reg Quinton's documentation on Solaris 7 SUID files and associated scripts).
    http://ist.uwaterloo.ca/security/howto/1999-04-21.html

    Solaris 2.6:
    http://ist.uwaterloo.ca/security/howto/2000-08-22.html

    Solaris 8:
    http://ist.uwaterloo.ca/security/howto/2000-08-17.html

  2. Titan's ziplock module:
    http://www.titan.org
  3. Example listing of SUID/SGID files on a Solaris 7 system:
    http://www.boran.com/security/sp/solaris/suid_sol7.txt

    Example listing of SUID/SGID files on a Solaris 8 system:
    http://www.boran.com/security/sp/solaris/suid_sol8.txt



Improving Disksuite Security

Disksuite is a tool bundled with Solaris that allows disks to be mirrored or gathered into RAID sets. This is useful and a nice feature to have in the system. The problem is that Disksuite uses RPC (specifically: two programs, rpc.metamhd and rpc.metad, which run from inetd). RPC is a protocol that one tries to avoid having on sensitive servers, such as those on the Internet or in a DMZ. RPC uses dynamic ports and provides no standard access control methods.

How can Disksuite security be improved?

  1. Don't run Disksuite. This is often my choice, mainly because of bad experiences with early versions of Disksuite 4-5 years ago. I don't forgive easily :-)
  2. Run Disksuite and RPC, but use Wietse Venema's RPCBIND with access control restricted to localhost. This option is discussed in the YASSP post-install documentation:
    http://www.yassp.org/after.html#RPC
  3. Run Disksuite, but stop RPC altogether. I didn't know this was possible until a recent discussion on the YASSP discussion list. The &quotmetad" services in inetd.conf can be disabled, with the following consequences:

Doug Hughes has documented tips and links on Disksuite, VxVM and the SparcStorage Array:
http://www.eng.auburn.edu/pub/mail-lists/ssastuff



Resources for Hardening Solaris

Over the last two years or so, several (free) useful tools and papers on hardening Solaris have appeared. Here we present a list with some comments.

My favorites are Titan and YASSP, but the idea is to provide a decent list of alternatives here, so you can learn from the various papers and pick an appropriate tool for your needs.
Titan and YASSP have different approaches and are useful in different situations. I would prefer YASSP for DMZ/bastion host hardening, but Titan is probably more useful for securing general workstations and multi-user servers. Also, each has features not found in the other.

Yassp (Yet Another Solaris Security Package)

YASSP
http://www.yassp.org
The goal is to install Solaris and have a good host security without having to spend hours in modification. According to Alan Paller, director of research at the SANS Institute, "When these scripts have been field tested, they will become the recommended solution for hardening Solaris systems and we will promote them widely."
YASSP is in final beta; the final release should be published very soon.

Hardening Solaris (Sean's articles on manual, Yassp and Jass based hardening)
http://www.boran.com/security/sp/Solaris_hardening.html  

Interview with Jean Chouanard (main developer behind YASSP)
http://www.boran.com/security/sp/interview_chouanard.html

Titan

The Titan project
http://www.fish.com/titan/
http://www.fish.com/titan/TITAN_documentation.html
Titan is a collection of programs, each of which either fixes or tightens one or more potential security problems with a particular aspect in the setup or configuration of a Unix system. Conceived and created by Brad Powell, it was written in Bourne shell, and its simple modular design makes it trivial for anyone who can write a shell script or program to add to it, as well as to completely understand the internal workings of the system.

Hardening Solaris - Compass Security Draft 0.82, by Ivan Butler 
http://www.csnc.ch/download/sources/Hardening-Solaris V0.82.pdf  
This PDF document provides a step-by-step tutorial to creating a Solaris system resistant to various methods of attack, based on the Titan scripts.

Sun

Sun's hardening tool, Jass
http://www.sun.com/blueprints/tools
Jass has a restrictive license and is still in beta. It was tested a few weeks back in Tip of the Week and didn't seem ready for prime time.

Sun's hardening documentation:

More Hardening Papers

SecurityFocus, list of Sun relevant articles
http://www.securityfocus.com/focus/sun/menu.html?fm=0&action=unfold

Lance Spitzner's white papers
http://ww.enteract.com/~lspitz/papers.html
These papers are useful and referenced by many people. Worth a read.

Securing Solaris Servers - A Checklist Approach, by Paul D. J. Vandenberg and Susan D. Wyess
http://www.usenix.org/sage/sysadmins/solaris/index.html#host

This material is excerpted from an internal U.S. Government document on web security, which the authors played leading roles in preparing. This material has been officially reviewed, and the authors have been granted permission to use this material in a non-official publication.

Hardening Solaris (Sean's articles on manual, Yassp and Jass based hardening)
http://www.boran.com/security/sp/Solaris_hardening.html  

tcp tuning under solaris, by Jens-S. Vöckler
http://www.rvs.uni-hannover.de/people/voeckler/tune/EN/tune.html

Security: How to Documents, Information Systems and Technology, University of Waterloo
http://ist.uwaterloo.ca/security/howto/

Wietse Venema's tools and papers (tcp wrapper, rpcbind/portmapper, postfix, Satan, ....)
ftp://ftp.porcupine.org/pub/security/index.html

Solaris Security Guide, Sabernet
http://www.sabernet.net/papers/Solaris.html

Solaris Security Step by Step, from SANS
http://www.sans.org
This is available in paper or PDF form, and is quite useful, but not free.

Sunworld

Softpanorama University Pages: Solaris Hardening and Security
http://www.softpanorama.org/Security/sos.shtml
This site is an index to many Solaris security papers and tools

Securiteam: Hardening Solaris SPARC/x86 security for Firewall usage - a step by step guide
http://www.securiteam.com/unixfocus/Hardening_Solaris_SPARC_x86_security_for_Firewall_usage_-_a_step_by_step_guide.html

More Hardening Tools

New Approaches to Making Solaris More Secure, by Rich Teer (including hardening scripts)
http://www.sysadminmag.com/supplement/web_feature1.shtml

Securing Solaris, by Ido Dubrawsky
http://www.sysadminmag.com/supplement/913secsol.shtml

Casper Dik's fixmode (improves Solaris file permissions)
ftp.wins.uva.nl:/pub/solaris

Chris Calabrese's Harden script
ftp://ftp.freebird.org/unixware/freebird/internet/systools/harden  

Alberto Begliomini's SECUR
ftp://ftp.coldstone.com/secur



C2, TFTPD

Back in August we had a brief look at Solaris C2 auditing / the 'BSM' module, and presented some scripts which make examining the audit logs easier. A new C2 article has been published in the special November supplement of Sysadmin magazine:

Implementing C2 Auditing in the Solaris Environment, by Kevin Wenchel and Stephen Michaels
http://www.sysadminmag.com/supplement/913c2.shtml

This article is easy to read and useful, and references a GUI developed by the authors.

The GUI, "BSM Event Viewer v1.1," is implemented in Perl/Tk and is easily compiled with gcc and Perl5. However, it crashed sometimes and I was unable to persuade it to open all the files in /var/audit or open individual log files. The author, Kevin Wenchel, thinks the problem might be with my X server (eXceed on NT4), so it may well work for you. A new release with some fixes is also on the way.

Despite these problems, the article is worth a read, and this space will be updated when we get the tools working.


TFTPD

On a different topic, I was on the hunt for a free TFTP server for Solaris with sources and came across utftpd v0.2.4.
http://www.ohse.de/uwe/software/utftpd.html

This server is interesting. It can be chrooted and run as a non-root user, and contains IP-level access control for clients (each client IP can be assigned create/read/write rights) and support automatic pushing and pulling of files from a version control system like SCCS or RCS (useful for managing router configuration files).

One minor change was needed to get it to compile on Solaris 8. In utftpd.c, change:
   pid_t pi=waitpid(WAIT_ANY,0,WNOHANG);
to
   pid_t pi=waitpid(0,0,WNOHANG);



YASSP beta#13

This week, I would like to shamelessly plug a project that I contribute to, which is a major benefit to the Solaris community. I would like to see many more of you involved in testing/reviewing/giving feedback. Below is an overview of YASSP, its features, disadvantages and advantages.

Now that the licensing issues have been sorted out, YASSP beta 12 and 13 were released this week, with the hope for releasing V1.0 very soon. The changes in the last two betas are documented in the previous section.

What is needed now, are testers for Solaris 2.6/2.7 SPARC and especially on Intel (nudge, nudge...).

YASSP Overview

Jean Chouanard of Xerox and a host of Solaris experts have developed the YASSP (Yet Another Solaris Security Package) [1] scripts for hardening Solaris, which includes many useful precompiled security tools.

The first version appeared in summer 1999 and adhered pretty closely to the SANS Solaris Hardening Guide. We now have an improved YASSP beta#12 (Nov. 2000) that now goes further and is a final release candidate. It's been tested by admins, has received much input from experts and tries to pull together all known issues of Solaris hardening into one bundle. YASSP includes scripts and binaries for key tools and allows individual tuning, so that it can be used for bastion hosts, servers and workstations.

The core hardening features are in the SECclean package in YASSP:

There is also a tarball available in addition to SECclean. This tarball also includes "GNUgzip GNUrcs OPENssh WVtcpd PARCdaily" i.e., binary packages for SPARC and X86. The tools are installed in /opt/local, except tripwire, which goes in /secure/tripwire.

YASSP installation logs:


YASSP Advantages


YASSP Disadvantages


YASSP - Suggested Improvements


Yassp - References

[1] www.yassp.org YASSP (yet another solaris security package) for Solaris 2.6/7/8
The list of SECclean hardening actions is documented at www.yassp.org/internal.html
Express installation guide: www.yassp.org/express.html
The developers email list is archived at www.theorygroup.com/Archive/YASSP
www.boran.com/security/sp/interview_chouanard.html 
[10] Casper Dik's fix-modes script contains a huge number of file permission improvements for most Solaris versions. ftp.wins.uva.nl:/pub/solaris/fix-modes.tar.gz
Jens Vöckler's nettune script www.rvs.uni-hannover.de/people/voeckler/tune/EN/tune.html
See also the local copy nettune.
[11] The Titan Project  www.titan.org
[12] Hardening Solaris with YASSP beta#13 (draft)
www.boran.com/sp/security/Solaris_hardening3.html
[13] Example YASSP beta#13 installation output
www.boran.com/sp/security/solaris/yassp_install.txt



Improving "usermod"

In a discussion on "focus-sun" this week, Bennet Todd suggests a useful way of temporarily locking a user on Solaris, without destroying his/her password:

On other OSes, the command "usermod -L acct" prepends a "!" to the encrypted password field, and -U removes the "!". If you don't have something like that handy, it's easy enough to do with perl, something like:
perl -pi.bak -e 's/^'$user':/'$user':!/' /etc/shadow

The complementary command to unlock an account would be the very similar:
perl -pi.bak -e 's/^'$user':!/'$user':/' /etc/shadow

The usermod(1M) command is normally used to change a user's fields in /etc/passwd and /etc/shadow, e.g., home directory, shell, expiry date, login name. The idea is to write a script of the same name that accepts additional options "-L" and "-U" and otherwise calls the standard usermod. The script must be in the path before the real usermod command.

Note: This script directly edits /etc/shadow, so it won't work for NIS+ and must be run as root (hence permissions of 700 are suggested).

I've written an improved script that checks that the account really exists and confirms that the change has been made.

#!/bin/sh
#
# used wrapper:
#    make sure this is in your path before /usr/sbin
#
# If 1st argument is -L, locks users account by prepending
#    a '!' to the encrypted password in /etc/shadow.
# If 1st argument is -U, remove the '!'
# Otherwise call the normal Solaris 'usermod' tool.
#
# We check to make sure there is a valid entry in the shadow file first.
# A backup of the shadow is also made in /etc/shadow.bak

shadow="/etc/shadow";
usermod="/usr/sbin/usermod";

case "$1" in
-L)
  if [ `egrep -c "^$2:" $shadow` = 1 ] ; then
    perl -pi.bak -e 's/^'$2':/'$2':!/' $shadow
    echo "Account $2 now blocked."
  else
    echo "$0 Error: invalid account '$2'"                                       
  fi;;

-U)
  if [ `egrep -c "^$2:" /etc/shadow` = 1 ] ; then
    perl -pi.bak -e 's/^'$2':!/'$2':/' $shadow
    echo "Account $2 re-enabled."
  else
    echo "Error: invalid account '$2'"
  fi;;

*) $usermod $*;;

esac



"boot -r"

Solaris recognizes disk devices dynamically when booting and loads appropriate drivers. New devices are not automatically recognized (presumably for faster booting).

If a new disk is added, the system is typically rebooted and told to search for new devices. This can be done by several methods:

There is an easier way, even without rebooting.

Tell Solaris to update /devices and then create appropriate device links for disks (and possibly tapes or serial ports):

drvconfig
disks
ports
tapes

Solaris 8 has a new command for managing devices, "devfsadm."

devfsadm(1M) maintains the /dev and /devices namespaces. It replaces the previous suite of devfs administration tools including drvconfig(1M), disks(1M), tapes(1M), ports(1M), audlinks(1M), and devlinks(1M).

devfsadmd(1M) is the daemon version of devfsadm(1M). The daemon is started by the /etc/rc* scripts during system startup and is responsible for handling both reconfiguration boot processing and updating /dev and /devices in response to dynamic reconfiguration event notifications from the kernel.

So this one command can be used for reconfiguring on Solaris 8:

devfsadm

Note: On my hardened Solaris 8 systems, the daemon "devfsadmd" is disabled without any obvious ill effects. It is probably necessary on systems like the Starfire which support dynamic changing of hardware.



Solaris on Headless PCs

Sun system administrators often manage the consoles of servers via serial lines: almost everything can be done remotely, including OS installations. This feature is possibly due to the powerful boot prom monitor included in Sun's SPARC systems (an advantage that Sun hardware has over all others I've seen so far).

The fun starts when trying to do the same with PCs running Solaris. The PC does not have an intelligent prom, so what Sun did was to implement the boot prom monitor in Software (it is called the Device Configuration Assistant). Booting a Solaris PC has several key steps:

  1. The PC BIOS decides what devices will be used for booting, in what order and whether certain devices are disabled.
  2. The SCSI controller (if there is one) can be configured to insist on its own boot device.
  3. The boot disk has been now selected and activated.
  4. Device Configuration Assistant detects devices and makes them visible to the boot manager. If Esc is pressed when this is booting, devices can be changed as can default boot order, boot disks, etc., via a menu-based interface. It is installed in the first few disk cylinders, in a separate partition (/boot).
  5. Boot Manager: Allow options to be given to Solaris on booting, or using the boot interpreter.
  6. Solaris

It is possible to get PCs to use the serial port A (COM1) as their console — you just need a null modem cable and a bit of patience. . . .

On Solaris 8, the following commands divert console input and output to the first COM port and ignore "carrier detect" signals (otherwise the machine won't reboot unless you are connected to the serial line). The Device Configuration Assistant can also be used to set these options.

eeprom ttya-ignore-cd=true
eeprom input-device=ttya
eeprom output-device=ttya

Note: Apparently the console redirection can also be activated by editing /platform/i86pc/boot/solaris/bootenv.rc and adding: "setprop output-device com1", "setprop input-device com1".


Connection to the Serial Console

If connecting from another Solaris box, say on its second serial port:


Notes


References



The Dreaded Patches

Patch Strategies

Weaknesses are continually discovered in Solaris and third-party applications. Not only do the weaknesses pose threats, but the volume of weaknesses and patches can be a threat: If not managed carefully, they will consume too much time or they will be simply ignored.

The first problem is to be aware that weaknesses and/or patches actually exist. Possible strategies are:

  1. Get on all the mailing lists of organizations such as CERT/First, vendors like Sun, and especially Bugtraq. This can consume quite a lot of time.
  2. Get on the mailing list of an organization which produces regular summaries of weakness/patches and security news, for example SecurityFocus, SANS or maybe this Digest :-). If a regular source of reliable information on Solaris and the applications that you use can be found, it may save much time.
  3. Run a tool on important servers that checks the current patch level and compares it with the newest list of patches available from Sun: This is the ideal situation (it is discussed below in more detail), but third-party applications will not be covered.
  4. Check Sun's recommended patch bundles for changes every month or two: This requires little effort, but security is not as tight. Third-party weaknesses are not covered and the recommended bundles do not cover all security problems.

How do you decide whether a weakness is worth patching?


Patch Tools

Tools to help find patches relevant to your systems [1]


References [1]

Reg Quinton/Bruce Barnett's CheckPatches, CheckPatches.cron, GetApplyPatch, GetApplyPatch.cron scripts
http://ist.uwaterloo.ca/~reggers/drafts/

SunSolve sunsolve.sun.com (Patchdiag & XREF file)

A SunSolve Patch Primer
sunsolve.sun.com/pub-cgi/show.pl?target=content/content1

SecurityFocus Vulnerability calculator
SecurityFocus.com/focus/sun/form.html

Casper Dik's FastPatch
ftp://www.wins.uva.nl/pub/solaris/auto-install/

Joe Shambin's Patchreport
ftp://x86.cs.duke.edu/pub/PatchReport/index.html



Seán Boran (sean at boran.com) is an IT security consultant based in Switzerland and the author of the online IT Security Cookbook.