If you run into a system that you can't access, but is available on the network, and have no idea what type of system that is, then there are few tricks you can use to determine the type of system remotely.

The first one, is by looking at the TTL (Time To Live), when doing a ping to the system's IP address. For example, a ping to an AIX system may look like this:

# ping 10.11.12.82
PING 10.11.12.82 (10.11.12.82) 56(84) bytes of data.
64 bytes from 10.11.12.82 (10.11.12.82): icmp_seq=1 ttl=253 time=0.394 ms
...

TTL (Time To Live) is a timer value included in packets sent over networks that tells the recipient how long to hold or use the packet before discarding and expiring the data (packet). TTL values are different for different Operating Systems. So, you can determine the OS based on the TTL value. A detailed list of operating systems and their TTL values can be found here. Basically, a UNIX/Linux system has a TTL of 64. Windows uses 128, and AIX/Solaris uses 254.

Now, in the example above, you can see "ttl=253". It's still an AIX system, but there's most likely a router in between, decreasing the TTL with one.

Another good method is by using nmap. The nmap utility has a -O option that allows for OS detection:

# nmap -O -v 10.11.12.82 | grep OS
Initiating OS detection (try #1) against 10.11.12.82 (10.11.12.82)
OS details: IBM AIX 5.3
OS detection performed.

Okay, so it isn't a perfect method either. We ran the nmap command above against an AIX 7.1 system, and it came back as AIX 5.3 instead. And sometimes, you'll have to run nmap a couple of times, before it successfully discovers the OS type. But still, we now know it's an AIX system behind that IP.

Another option you may use, is to query SNMP information. If the device is SNMP enabled (it is running a SNMP daemon and it allows you to query SNMP information), then you may be able to run a command like this:

# snmpinfo -h 10.11.12.82 -m get -v sysDescr.0
sysDescr.0 = "IBM PowerPC CHRP Computer
Machine Type: 0x0800004c Processor id: 0000962CG400
Base Operating System Runtime AIX version: 06.01.0008.0015
TCP/IP Client Support  version: 06.01.0008.0015"

By the way, the example for SNMP above is exactly why UNIX Health Check generally recommends to disable SNMP, or at least to dis-allow providing such system information trough SNMP by updating the /etc/snmpdv3.conf file appropriately, because this information can be really useful to hackers. On the other hand, your organization may use monitoring that relies of SNMP, in which case it needs to be enabled. But then you stil have the opportunity of changing the SNMP community name to something else (the default is "public"), which also limits the remote information gathering possibilities.

How to fix subscription-manager error "Too many content sets for certificate Red Hat Enterprise Linux Server" using RHN and be able to revert back to Red Hat Subscription Management after updating.

Step 1: Clean up the subscription-manager if needed:

# subscription-manager unsubscribe --all
# subscription-manager unregister
# subscription-manager clean

Step 2: Register to Red Hat Network (RHN) using rhn_register:

# rhn_register

Note: You will need your RH login and password to complete the wizard.

Step 3: Validate RHN registration of the system:

# yum repolist

Note: Look at Loaded plugins in the output and "rhnplugin" should be listed.

Step 4: Update subscription-manager* and python-rhsm* packages: # yum list updates subscription-manager* python-rhsm* Note: The output may vary depending on your system and installed packages.

Example output below:

Updated Packages
python-rhsm.x86_64 1.12.5-2.el6 rhel-x86_64-server-6
subscription-manager.x86_64 1.12.14-9.el6_6 rhel-x86_64-server-6
subscription-manager-firstboot.x86_64 1.12.14-9.el6_6 rhel-x86_64-server-6
subscription-manager-gnome.x86_64 0.99.19.4-1.el6_3 rhel-x86_64-server-6
# yum update subscription-manager* python-rhsm*

Note: Answer the questions when prompted. Validate the updates were applied successfully by examining the output.

Step 5: Unregister from RHN in preparation to register with subscription-manager:

1. In the online Red Hat Portal, login.
2. Access Subscription Management.
3. Access RHN Classic Management -> All Registered Systems.
4. Click on System Entitlements (you need to see check boxes next to systems).
5. Select the check box next to the system you are working on.
6. Click the "Unentitle" button at bottom middle of page.
7. Validate the entitlement has been removed for the system.
8. Perform the below command on the system's CLI:

   # rm /etc/sysconfig/rhn/systemid

Step 6: Register system with subscription-manager:

Note: Validate that no subscriptions are showing active.

# subscription-manager list --available

Note: A message similar to below should be displayed.

This system is not yet registered. Try 'subscription-manager register --help' for more information.

Register the system using your credentials to RHSM:

# subscription-manager register --username=xxxxxx --password='xxxxxx'

Note: You will need your Red Hat Portal Username and Password for the account the system will be registered under. Make note of the ID that the system will be registered when this command returns.

Validate that the subscription-manager plugin is loaded

# yum repolist

Look at Loaded plugins in the output where "subscription-manager" should be listed.

Validate that subscriptions are showing available now:

# subscription-manager list --available

Validate the Subscription Name, SKU, Contract, Account and Pool ID are showing up correctly. Make note of the "Pool ID" that will be required to subscribe in the next task. Register the system using one of the pools above:

# subscription-manager subscribe --pool='[POOL_ID_Number]'

Note: Where "[POOL_ID_Number]" should be obtained from the preceding task.

Make sure a message stating "Successfully attached a subscription for" the system is shown.

Step 7: Validate that the system is now consuming a subscription:

# subscription-manager list --consumed

Validate the Subscription Name, SKU, Contract, Account and Pool ID are correct.

# subscription-manager list

Note: The Status should show "Subscribed".

Step 8: Validate in Red Hat Portal that the new system shows up as well.

In Red Hat Portal:

1. In the online Red Hat Portal, login.
2. Access Subscription Management.
3. Access Red Hat Subscription Management -> Subscriber Inventory -> Click on Systems.
4. Examine the Systems inventory to validate the new system is now visible and shows a subscription attached.

Here's how you can set up your /etc/profile in order to create a separate shell history file for each user and each login session. This is very useful when you need to know who exactly ran a specific command at a point in time. For Red Hat Linux, put the updates in either /etc/profile or /etc/bashrc.

Put this in /etc/profile on all servers:

# HISTFILE
# execute only if interactive
if [ -t 0 -a "${SHELL}" != "/bin/bsh" ]
then
 d=`date "+%H%M.%m%d%y"`
 t=`tty | cut -c6-`
 u=`who am i | awk '{print $1}'`
 w=`who -ms | awk '{print $NF}' | sed "s/(//g" | sed "s/)//g"`
 y=`tty | cut -c6- | sed "s/\//-/g"`
 mkdir $HOME/.history.$USER 2>/dev/null
 export HISTFILE=$HOME/.history.$USER/.sh_history.$USER.$u.$w.$y.$d
 find $HOME/.history.$USER/.s* -type f -ctime +91 -exec rm {} \; 2>/dev/null

 H=`uname -n | cut -f1 -d'.'`
 mywhoami=`whoami`
 if [ ${mywhoami} = "root" ] ; then
  PS1='${USER}@(${H}) ${PWD##/*/} # '
 else
  PS1='${USER}@(${H}) ${PWD##/*/} $ '
 fi
fi

# Time out after 60 minutes
# Use readonly if you don't want users to be able to change it.
# readonly TMOUT=3600
TMOUT=3600
export TMOUT

When using ksh, put this in /etc/environment, to turn on time stamped history files:

# Added for extended shell history
EXTENDED_HISTORY=ON

When using bash, put this in /etc/bashrc, to enable time-stamped output when running the "history" command:

HISTTIMEFORMAT='%F %T '; export HISTTIMEFORMAT

This way, *every* user on the system will have a separate shell history in the .history directory of their home directory. Each shell history file name shows you which account was used to login, which account was switched to, on which tty this happened, and at what date and time this happened.

Shell history files are also time-stamped internally. For AIX, you can run "fc -t" to show the shell history time-stamped. For Red Hat, you can run: "history". Old shell history files are cleaned up after 3 months, because of the find command in the example above. Plus, user accounts will log out automatically after 60 minutes (3600 seconds) of inactivity, by setting the TMOUT variable to 3600. You can avoid running into a time-out by simply typing "read" or "\" followed by ENTER on the command line, or by adding "TMOUT=0" to a user's .profile, which essentially disables the time-out for that particular user.

One issue that you now may run into on AIX, is that because a separate history file is created for each login session, that it will become difficult to run "fc -t", because the fc command will only list the commands from the current session, and not those written to a different history file. To overcome this issue, you can set the HISTFILE variable to the file you want to run "fc -t" for:

# export HISTFILE=.sh_history.root.user.10.190.41.116.pts-4.1706.120210

Then, to list all the commands for this history file, make sure you start a new shell and run the "fc -t" command:

# ksh "fc -t -10"

This will list the last 10 commands for that history file.

If you have performend a RHEL 7 Linux Server installation and did not include Graphical User Interface (GUI) you can do it later directly from command line using yum command and selecting an appropriate installation group. To list all available installation groups on Redhat 7 Linux use:

# yum group list

From the above list select Server with GUI installation group:

# yum groupinstall 'Server with GUI'

Just because gnome desktop environment is a default GUI on RHEL 7 linux system the above command will install gnome. Alternatively, you can run the below command to only install core GNOME packages:

# yum groupinstall 'X Window System' 'GNOME'

Once the installation is finished, you need to change system's runlevel to runlevel 5. Changing runlevel on RHEL 7 is done by use of systemctl command. The below command will change runlevel from runlevel 3 to runelevel 5 on RHEL 7:

# systemctl enable graphical.target --force

Depending on your previous installations you may need to accept Redhat License after you reboot your system. Once you boot to your system you can check GNOME version using:

# gnome-shell --version

Source: http://linuxconfig.org/install-gnome-gui-on-rhel-7-linux-server.

This is a short procedure that will tell you how to set up a local repository (repo) for use by the yum command, to install packages from onto your system. In this procedure, we assume you have the RHEL installation DVD inserted into your virtual or physical drive.

Mount the drive:

# mkdir /cdrom
# mount /dev/cdrom /cdrom

Then create the repo file in /etc/yum.repos.d, called local.repo:

# cd /etc/yum.repos.d
# vi local.repo
[local]
name=Local Repo
baseurl=file:////cdrom
enabled=1
gpgcheck=0
protect=1

From now on you can use this local repository to install software, such as wireshark:

# yum install wireshark

Here's a procedure how you can add additional swap space to a running RHEL system.

This procedure assumes you will want to add 8 Gigabytes of swap space, and we will be using LVM to do so. To get information from Red Hat on recommended swap space sizes, take a look here: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/Deployment_Guide/ch-swapspace.html.

First start by checking what the current swap space size is, by using the free command:

# free -m -t
             total       used       free     shared    buffers     cached
Mem:        129013     124325       4688          9        173      97460
-/+ buffers/cache:      26691     102322
Swap:        16383       8057       8326
Total:      145397     132382      13015

This particular system has 16 GB of swap space (look in the "total" column next to "Swap:"). Using the -m option with the free command displays the memory values in megabytes. Using the -t option will provide the totals.

You can also see that the system has used 8057 MB of it's swap space, almost half of the swap space available.

Then, figure out how the current swap spaces are configured now:

# cat /proc/swaps
Filename                         Type            Size    Used    Priority
/dev/dm-1                        partition       8388604 8262740 -1
/dev/dm-8                        partition       8388604 0       -2

This shows that there are 2 paging spaces of 8 GB each. To increase the swap space on the system, we'll add another swap space of 8 GB, so the total swap space will go up to 24 GB.

To get a view of what logical volumes exist on the system, use the dmsetup command:

# dmsetup ls
rootvg00-optlv00        (253:7)
rootvg00-tmplv00        (253:3)
rootvg00-varlv00        (253:2)
rootvg00-homelv00       (253:6)
rootvg00-rootlv00       (253:0)
rootvg00-usrlocallv00   (253:5)
rootvg00-swaplv01       (253:8)
rootvg00-usrlv00        (253:4)
rootvg00-swaplv00       (253:1)

This shows that there are 2 logical volumes, swaplv00, and swaplv01. We'll create swaplv02 as the third swap space on the system.

Another good way to see the same information, is by using the lvs command:

# lvs 2>/dev/null
  LV           VG       Attr       LSize
  homelv00     rootvg00 -wi-ao---- 10.00g
  optlv00      rootvg00 -wi-ao----  8.00g
  rootlv00     rootvg00 -wi-ao----  2.00g
  swaplv00     rootvg00 -wi-ao----  8.00g
  swaplv01     rootvg00 -wi-ao----  8.00g
  tmplv00      rootvg00 -wi-ao----  5.00g
  usrlocallv00 rootvg00 -wi-ao----  1.00g
  usrlv00      rootvg00 -wi-ao----  5.00g
  varlv00      rootvg00 -wi-ao----  4.00g

This gives you the information that the logical volumes have been created in the rootvg00 volume group. We'll create the new swap space in the same volume group, using the lvcreate command:

# lvcreate -n swaplv02 -L 8G rootvg00
  Logical volume "swaplv02" created

Using the -n option of the lvcreate command, you can specify the name of the logical volume. The -L option specifies the size (in this case 8G), and you end the command with the volume group name.

Next, you'll have to tell RHEL that the new logical volume is to be formatted for swap space usage:

# mkswap /dev/rootvg00/swaplv02
Setting up swapspace version 1, size = 8388604 KiB
no label, UUID=c9be43f7-c473-45ae-ba13-c1e09af2d95e

Then, you'll have to add an entry to /etc/fstab, so the system knows to re-use the swap space after a system reboot:

# grep swap /etc/fstab
/dev/mapper/rootvg00-swaplv00 swap     swap    defaults        0 0
/dev/mapper/rootvg00-swaplv01 swap     swap    defaults        0 0
/dev/mapper/rootvg00-swaplv02 swap     swap    defaults        0 0

Finally, activate the new swap space using the swapon command:

# swapon -v /dev/rootvg00/swaplv02 swapon on /dev/rootvg00/swaplv02 swapon: /dev/mapper/rootvg00-swaplv02: found swap signature: version 1, page-size 4, same byte order swapon: /dev/mapper/rootvg00-swaplv02: pagesize=4096, swapsize=8589934592, devsize=8589934592

To validate that the new swap space is available on the system, use the free command again, and you may also review /proc/swaps:

# free -m -t
             total       used       free     shared    buffers     cached
Mem:        129013     121344       7669          9        175      95575
-/+ buffers/cache:      25593     103420
Swap:        24575       8109      16466
Total:      153589     129453      24136
# cat /proc/swaps
Filename                         Type            Size    Used    Priority
/dev/dm-1                        partition       8388604 8303856 -1
/dev/dm-8                        partition       8388604 0       -2
/dev/dm-9                        partition       8388604 0       -3

That's it; you're done!

On Linux, you sometimes may run into an issue where you can't change permissions of a file, even though you're root, and you have access. For example:

# ls -asl authorized_keys
8 -rw------- 1 root root 6325 Sep 17 02:48 authorized_keys
# chmod 700 authorized_keys
chmod: changing permissions of `authorized_keys': Operation not 
permitted
# whoami
root

This is usually caused by the Extendef File System Attributes, especially if package e2fsprogs is installed. Two commands that will come in handy here are /usr/bin/chattr and /usr/bin/lsattr.

The most common attributes are:

• A - When the file is accessed the atime record is not modified. This avoids a certain amount of disk I/O.
• a - When this file is opened, it is opened in append only mode for writing.
• i - This file cannot be modified, renamed or deleted.

For example:

# lsattr authorized_keys
----i-------- authorized_keys

This shows that the immutable flag (i) is in place on the file, and thus the reason why the file can't be modified. To remove it, use chattr:

# chattr -i authorized_keys
# lsattr authorized_keys
------------- authorized_keys

Now any commands to modify the file, will work:

# chmod 700 authorized_keys

VIM on many different types of installations will create both swap files and backup files.

How to disable VIM swap and backup files:

Go into your _vimrc file. Add these lines to the bottom:

set nobackup
set nowritebackup
set noswapfile

On Linux, you can use the tmpfs to create a RAM disk:

# mkdir -p /mnt/tmp
# mount -t tmpfs -o size=20m tmpfs /mnt/tmp

This will create a 20 Megabyte sized RAM file system, mounted on /mtn/tmp. If you leave out the "-o size" option, by default half of the memory will be allocated. However, the memory will not be used, as long as no data is written to the RAM file system.

Sudosh is designed specifically to be used in conjunction with sudo or by itself as a login shell. Sudosh allows the execution of a root or user shell with logging. Every command the user types within the root shell is logged as well as the output.

This is different from "sudo -s" or "sudo /bin/sh", because when you use one of these instead of sudosh to start a new shell, then this new shell does not log commands typed in the new shell to syslog; only the fact that a new shell started is logged.

If this newly started shell supports commandline history, then you can still find the commands called in the shell in a file such as .sh_history, but if you use a shell such as csh that does not support command-line logging you are out of luck.

Sudosh fills this gap. No matter what shell you use, all of the command lines are logged to syslog (including vi keystrokes). In fact, sudosh uses the script command to log all key strokes and output.

Setting up sudosh is fairly easy. For a Linux system, first download the RPM of sudosh, for example from rpm.pbone.net. Then install it on your Linux server:

# rpm -ihv sudosh-1.8.2-1.2.el4.rf.i386.rpm
Preparing...  ########################################### [100%]
   1:sudosh   ########################################### [100%]

Then, go to the /etc file system and open up /etc/sudosh.conf. Here you can adjust the default shell that is started, and the location of the log files. Default, the log directory is /var/log/sudosh. Make sure this directory exists on your server, or change it to another existing directory in the sudosh.conf file. This command will set the correct authorizations on the log directory:

# sudosh -i
[info]: chmod 0733 directory /var/log/sudosh

Then, if you want to assign a user sudosh access, edit the /etc/sudoers file by running visudo, and add the following line:

username ALL=PASSWD:/usr/bin/sudosh

Now, the user can login, and run the following command to gain root access:

$ sudo sudosh
Password:
# whoami
root

Now, as a sys admin, you can view the log files created in /var/log/sudosh, but it is much cooler to use the sudosh-replay command to replay (like a VCR) the actual session, as run by the user with the sudosh access.

First, run sudosh-replay without any paramaters, to get a list of sessions that took place using sudosh:

# sudosh-replay
Date       Duration From To   ID
====       ======== ==== ==   ==
09/16/2010 6s       root root root-root-1284653707-GCw26NSq

Usage: sudosh-replay ID [MULTIPLIER] [MAXWAIT]
See 'sudosh-replay -h' for more help.
Example: sudosh-replay root-root-1284653707-GCw26NSq 1 2

Now, you can actually replay the session, by (for example) running:

# sudosh-replay root-root-1284653707-GCw26NSq 1 5

The first paramtere is the session-ID, the second parameter is the multiplier. Use a higher value for multiplier to speed up the replay, while "1" is the actual speed. And the third parameter is the max-wait. Where there might have been wait times in the actual session, this parameter restricts to wait for a maximum max-wait seconds, in the example above, 5 seconds.

For AIX, you can find the necessary RPM here. It is slightly different, because it installs in /opt/freeware/bin, and also the sudosh.conf is located in this directory. Both Linux and AIX require of course sudo to be installed, before you can install and use sudosh.