Less known Solaris features: The curious case of the /tmp in Solaris
This article isn’t really about a feature, it’s about a directory and its misuse. Furthermore it’s an article about different default configurations, that lead to misunderstandings. This is a pretty old hat for experienced Solaris admins (many of them learned it the hard tour), but it seems to be totally unknown to many admins new to the business or for people switching from Linux to Solaris, as many distribution are configured in a different way per default. A reader of my blog just found a 2GB .iso in /tmp on a Solaris system and that’s not really a good idea. A few days ago, a user in twitter had vast problems with memory usage on a system which boiled down to a crowed /tmp.
tmpfs and its usage
In Linux the /tmp directory is a normal directory. Most often this is just a part of the partition containing root. Many admins consider /tmp as an additional home directory and put data on it they want to use later on. When you really think about the idea behind /tmp is meant as a scratch space, where an application can put some data to process it later. Many applications use this /tmp space to write temporary data to.
Solaris introduced something called tmpfs many years ago, It’s a memory based file system. Linux has something with similar functionality and the same name. It´s just used differently in the default configuration of many distributions and there are a vast amount of articles in the web that suggests to configure /tmp in Linux to use the tmpfs filesystem as this can give you advantages for example with a Mysql for example. (when there is a reasonable amout of SELECTs opting for filesort)
The tmpfs is not a ramdisk. A ram disk just resides in the RAM, a tmpfs resides in the virtual memory, thus it uses the swap when the memory is needed for something else. Furthermore it doesn’t have a predefined size, albeit you can define a maximum size for the tmpfs to ensure that someone who writes into the /tmp can’t eat away all your virtual memory.
It’s called tmpfs because everything you write into it is temporary, the next boot or unmount will kill all the files on it. When you look at the mount table of a Solaris System you will recognize, that the usual locations for such temporary files are mounted tmpfs:
jmoekamp@a380:/var$ mount | grep "swap"
/etc/svc/volatile on swap read/write/setuid/devices/xattr/dev=4f00001 on Fri Jul 31 06:33:33 2009
/tmp on swap read/write/setuid/devices/xattr/dev=4f00002 on Fri Jul 31 06:34:14 2009
/var/run on swap read/write/setuid/devices/xattr/dev=4f00003 on Fri Jul 31 06:34:14 2009Keeping these file systems in virtual memory is a reasonable choice. The stuff in this directory is normally stale after a reboot, most of the time the files are many, but rather small and putting them on disk would just eat away your IOPS budget on your boot disks. As the file system resides in memory, it’s much faster and that really helps on jobs with many small files. A good example is compiling software when you use /tmp on as the TMPDIR.
All this advantages come with a big disadvantage, when you are not aware of the nature of the /tmp directory. I assume, you already know why using /tmp for storing ISOs is a bad idea. It eats away your memory and later on your swap. And for all the experienced admins: When someone has memory problems, ask at first about the /tmp directory, we tend to forget about this, as we’ve learned this lesson a long time ago and thus don’t think about this problem. When you really need a temporary place to store data in it for a while you should use /var/tmp. This is a directory on a normal disk based filesystem and thus its content it boot persistent.
Configuring the maximum size of the tmpfs
When you want to enforce a limit on the size of a tmpfs this pretty easy task . You can do this with a mount option. Let’s assume you want to provide the /var/application_scratch pad directory for an application, but you want to be sure, that this application can’t ruin your day by eating away all your memory:
jmoekamp@a380:/var# mkdir /var/application_scratchpad
jmoekamp@a380:/var# mount -f tmpfs -o size=128m swap /var/application_scratchpad/
jmoekamp@a380:/var# mount | grep "application"
/var/application_scratchpad on swap read/write/setuid/devices/xattr/size=128m/dev=4f00004 on Fri Jul 31 09:32:26 2009When you want to make a boot persistent change to the maximum size of the /tmp directory, you have to configure this in the /etc/vfstab:
swap - /tmp tmpfs - yes size=512mConclusion
I hope it´s now clear, why you shouldn´t use /tmp as a place for storing big files, at least move them directly somewhere else, when you use /tmp as the target directory to ssh a file to a system. At the other side the /tmp in virtual memory gives you some interesting capabilities to speed up your applications. As usual, everything has two sides: Making it default in Solaris gives you speedups per default. But when you are unaware of this default situation your virtual memory may be used for collection of videos ;) Obviously the same is valid, when you configure our Linux system with a tmpfs based /tmp and don’t tell it to your fellow admins.