How to use Intel Optane Memory for SSD Caching
However, if I can disable this process / clean the cache once after that, I think my system will speed up to some extent. Correct me if I am wrong here. I have also tried Bleachbit memory cleaning, but it doesn't seem to clean the memory cache properly. Also, the feature is still in an experimental stage. RAM or Random Access Memory stores program data and important machine code that are current being used by your computer. So basically RAMDISK is used as a cache here. In this article, I will show you how to create and use RAMDISK on Ubuntu 18.04 LTS. To watch a video of this process see video below: Creating a RAMDISK: Creating a RAMDISK on.
Recently Michael from Opvizor ordered a new laptop - a HP ProBook 650 G5. And here comes his story to enable Intel Optane using Ubuntu.
Addicted to Linux, the decision was clear to purge Windows and to setup Ubuntu, currently 18.04 LTS.
But wait:
The machine is equiped with an 'INTEL® OPTANE™ MEMORY H10 WITH SOLID STATE STORAGE'.
**What is that?
**Ask Google helped:
It is a solid state disk with an additional non volatile cache memory. That cache is build up of Optane memory, which is slower than DRAM, but faster than flash memory, and non volatile too.
Its easy for Windows users - there is the preinstalled Intel RST-Driver, which setups the Optane memory as cache for the SSD. But in Linux? No driver support from Intel for that.
However, using the Optane Memory isn't that hard. Many people on the net suggest to use it as an independent, fast, but also small disk.
But my workloads vary, and I don't want to waste time deciding what to put on Optane and what to put on flash. So I wanted to use the Optane Memory as cache too, like the Windows people do.
Optane Memory in Linux
On the physical layer, the device splits up into two independent nvme devices:
These devices are mapped as /dev/nvme0 and /dev/nvme1, and because of nvme technology, the usable blockdevices are /dev/nvme0n1 and /dev/nvme1n1 (aka nvme namespaces).
So the first step is to disable the BIOS option 'Advanced / System options / Configure Storage Controller for Intel Optane'. Have a look in the option ROM of the controller too, to make sure there is no combined device anymore. These options are intended for the RST-driver to recognize a device constructed of the two components. But in Linux we will do that without RST, so no need for that options.
Be aware: Since there is no support for RST's caching with Optane memory in Linux, you can't dual boot Windows with RST caching configured, and Linux. Either turn of Optane in Windows or put Windows into a VM in the Linux OS. To make things not complicated, the following won't consider dual boot.
Installation
Step 1
Start with a conventional installation, use the complete SSD (/dev/nvme0n1) with LVM and LUKS Encryption. LVM is mandatory for the latter steps, encryption optional, but assumed to be enabled. So in the installer, these are the selections:
After installation, the resulting disk layout is quite simple:
nvme0n1p1 holds /boot/efi, nvme0n1p2 holds /boot, and nvme0n1p3 holds a LUKS container, which is used by LVM as physical device for the volume group ubuntu-vg. LVM delivers the logical volumes for swap and root.
Configure LUKS
Ubuntu Ram Cache Clear History
When you have successfully booted into your new system, the next step is to configure the LUKS encryption layer:
The purpose of /etc/crypttab is to hold a list of encrypted devices. This information is essential for booting the system. There should be already one line for the device nvme0n1p3_crypt . Now add the line for nvme1n1_crypt, using the UUID from the last step.
Of course, your UUIDs will be different. The discard option is not used for the Optane-based device, since Optane Memory works in a different way than flash memory and so won't need discards.
lvmcache
The cache is build with lvmcache, gives a compact explanation. Use the command:
That command given, you created a pool for the caching using the complete optane memory. Due to thin provisioning capabilities, this pool serves both caching and metadata device needed by the kernel driver dm-cache.
The cache is created for the logical volume root (holding your root filesystem). The cache mode is set to writeback, which is perfectly good since you use non volatile memory!
Do not reboot
And now: DON'T REBOOT, since the machine won't come up again! You have to do some additional tasks. Enable the universe-repositories and install the thin-provisioning-tools.
These will be needed to check the consistency of the cached device on boot.
script thin-provisioning-tools
Download the script thin-provisioning-tools and install it to the directory /usr/share/initramfs-tools/hooks:
Download thin-provisioning-tools
The script is based on this article, with support for lvmcache added. The purpose of this script is, to put the binaries of the thin-provisioning-tools onto the initial ramdisk, and to load the required kernel modules for caching.
Finish
These initial ramdisk(s) have support for:
- second LUKS device based on the Optane Memory
- the kernel drivers needed to run the cache
- the tools used to check the cached volume, needed for starting it
NOW you are save to reboot. Don't be afraid when asked twice for LUKS passwords: once for nvme0n1p3_crypt, and once for nvme1n1_crypt. Time tracker for mac.
Results
To get a quick overview of your cache usage, issue the command
You can repeat the command over time. You will see the cache filling with data, and you will see the efficiency going up when looking at the ratio hits/misses. E.g.
Additional notes
If you decide to remove the cache from your configuration, run
Typically, Ubuntu 18.04 creates a logical volume for swapping. Consider to replace this with a swapfile in your root filesystem. The logical volume swap is not cached with Optane memory, but the logical volume root is.
Like any other operating system, GNU/Linux has implemented a memory management efficiently and even more than that. But if any process is eating away your memory and you want to clear it, Linux provides a way to flush or clear ram cache.
Clear RAM Cache and Swap in Linux
How to Clear Cache in Linux?
Every Linux System has three options to clear cache without interrupting any processes or services.
- Clear PageCache only.
- Clear dentries and inodes.
- Clear PageCache, dentries and inodes.
Explanation of above command.
sync will flush the file system buffer. Command Separated by “;” run sequentially. The shell wait for each command to terminate before executing the next command in the sequence. As mentioned in kernel documentation, writing to drop_cache will clean cache without killing any application/service, command echo is doing the job of writing to file.
If you have to clear the disk cache, the first command is safest in enterprise and production as “..echo 1 > ….” will clear the PageCache only.
It is not recommended to use third option above “..echo 3 >” in production until you know what you are doing, as it will clear PageCache, dentries and inodes.
Is it a good idea to free Buffer and Cache in Linux that might be used by Linux Kernel?
When you are applying various settings and want to check, if it is actually implemented specially on I/O-extensive benchmark, then you may need to clear buffer cache. You can drop cache as explained above without rebooting the System i.e., no downtime required.
Linux is designed in such a way that it looks into disk cache before looking onto the disk. If it finds the resource in the cache, then the request doesn’t reach the disk. If we clean the cache, the disk cache will be less useful as the OS will look for the resource on the disk.
Moreover it will also slow the system for a few seconds while the cache is cleaned and every resource required by OS is loaded again in the disk-cache.
Now we will be creating a shell script to auto clear RAM cache daily at 2am via a cron scheduler task. Create a shell script clearcache.sh and add the following lines.
Set execute permission on the clearcache.sh file.
Clear Ram Cache Ubuntu 20.04
Now you may call the script whenever you required to clear ram cache.
Now set a cron to clear RAM cache everyday at 2am. Open crontab for editing.
Aerial for mac. Append the below line, save and exit to run it at 2am daily.
For more details on how to cron a job you may like to check our article on 11 Cron Scheduling Jobs.
Is it good idea to auto clear RAM cache on production server?
No! it is not. Think of a situation when you have scheduled the script to clear ram cache everyday at 2am. Everyday at 2am the script is executed and it flushes your RAM cache. One day for whatsoever reason, may be more than expected users are online on your website and seeking resource from your server.
At the same time scheduled script run and clears everything in cache. Now all the user are fetching data from disk. It will result in server crash and corrupt the database. So clear ram-cache only when required,and known your foot steps, else you are a Cargo Cult System Administrator.
How to Clear Swap Space in Linux?
If you want to clear Swap space, you may like to run the below command.
Also you may add above command to a cron script above, after understanding all the associated risk.
Now we will be combining both above commands into one single command to make a proper script to clear RAM Cache and Swap Space.
OR
After testing both above command, we will run command “free -h” before and after running the script and will check cache.
How to Clear Swap Space in Linux?
If you want to clear Swap space, you may like to run the below command.
Also you may add above command to a cron script above, after understanding all the associated risk.
Clear Cache Ubuntu Terminal
Now we will be combining both above commands into one single command to make a proper script to clear RAM Cache and Swap Space.
Clear Ram Cache Ubuntu
OR
After testing both above command, we will run command “free -h” before and after running the script and will check cache.
Linux Clear Memory Cache
References:
1.http://www.tecmint.com/clear-ram-memory-cache-buffer-and-swap-space-on-linux/