Apr 122020
Picture of Raspberry Pi 4 box and Raspberry Pi 4 board below box

If you’re worried about destroying your SD Cards, need some more space, or just want to learn something new, I’m going to show you how to use an NFS root for the Raspberry Pi 4.

When you use an NFS Root with your Raspberry Pi, it stores the entire root filesystem on a remote NFS export (think of it as a network filesystem share). This means you’ll have as much space as the NFS export, and you’ll probably see way faster performance since it’ll be running at 1Gb/sec instead of the speed of the SD Card.

This also protects your SD card, as the majority of the reading and writing is performed on the physical storage of the NFS export, instead of the SD card in the Pi which has limited reads and writes.

What you’ll need

To get started, you’ll need:

  • Raspberry Pi 4
  • Ubuntu or Raspbian for Raspberry Pi 4 Image
  • A small SD card for the Boot Partition (1-2GB)
  • SD card for the Raspberry Pi Linux image
  • Access to another Linux system (workstation, or a Raspberry Pi)

There are multiple ways to do this, but I’m providing instructions on the easiest way it was for me to do this with the resources I had immediately available.


To boot your Raspberry Pi 4 from an NFS root, multiple steps are involved. Below you’ll find the summary, and further down you’ll find the full instructions. You can click on an item below to go directly to the section.

The process:

  1. Write the Linux image to an SD Card
  2. Create boot SD Card for NFS Root
  3. Prep the Linux install for NFS Root
  4. Create the NFS Export
  5. Copy the Linux install to the NFS Export
  6. Copy and Modify the boot SD Card to use NFS Root
  7. Boot using SD Card and test NFS Root

See below for the individual instructions for each step.

Write the Linux image to an SD Card

First, we need to write the SD Card Linux image to your SD card. You’ll need to know which device your SD card will appear to your computer. In my case it was /dev/sdb, make sure you verify the right device or you could damage your current Linux install.

  1. Download Ubuntu or Raspbian for Raspberry Pi.
  2. unzip or unxz depending on distribution to uncompress the image file.
  3. Write the SD card image to SD card.
    dd if=imagename.img of=/dev/sdb bs=4M

You now have an SD Card Linux install for your Raspberry Pi. We will later modify and then copy this to the NFS root and boot SD card.

Create boot SD Card for NFS Root

In this step, we’re going to create a bootable SD card that contains the Linux kernel and other needed files for the Raspberry Pi to boot.

This card will be installed in the Pi, load the kernel, and then kick off the boot process to load the NFS root.

I previously created a post to create a boot partition layout for a Raspberry Pi. Please follow those instructions to complete this step.

Later on in this guide, you’ll be copying the boot partition from the SD Card Linux image, on to this newly created boot SD Card for the NFS Root.

Prep the Linux install for NFS Root

There’s a few things we have to do to prep the Ubuntu or Raspbian Linux install to be usable as an NFS Root.

  1. Boot the Raspbian or Ubuntu SD Card you create in the first step on your Raspberry Pi.
  2. Complete the first boot procedures. Create your account, and complete the setup.
  3. Enable and confirm SSH is working so you can troubleshoot.
  4. Install the NFS client files using the following command:
    apt install nfs-common
  5. Open the /etc/network/interfaces file, and add the following line so that the Pi only get’s an IP once during boot:
    iface eth0 inet manual
  6. Modify your /etc/fstab entries to reflect the NFS root and the new boot SD card as per below.

For step 6, we need to modify the /etc/fstab entry for the root fs. It is different depending on whether you’re using Ubuntu or Raspbian.

For Raspbian, your /etc/fstab should look like this:

proc /proc proc defaults 0 0
LABEL=boot /boot vfat defaults 0 2
NFS-SERVER-IP:/nfs-export/PI-Raspbian / nfs defaults 0 0

For Ubuntu, your /etc/fstab should look like this:

LABEL=system-boot /boot/firmware vfat defaults 0 2
/dev/nfs / nfs defaults 0 0

After you do this, the Linux SD image may not boot again if directly installed in the Raspberry Pi, so make sure you’ve made the proper modifications before powering it down.

Create the NFS Export

In my case I used a Synology DS1813+ as an NFS server to host my Raspberry Pi NFS root images. But you can use any Linux server to host it.

If you’re using a synology disk station, create a shared folder, disable the recycling bin, leave everything else default. Head over to the “NFS Permissions” tab and create an ACL entry for your PI and workstations. You can also add a network segment for your entire network (ex.″) instead of specifying individual IPs.

Screenshot of Synology Create NFS rule for ACL
Create an NFS ACL Rule for Synology NFS Access

Once you create an entry, it’ll look like this. Note the “Mount path” in the lower part of the window.

Screenshot of NFS Shared Folder Permissions and Mount Point on Synology NAS
NFS Permissions and Mount Path for NFS Export

Now, if you’re using a standard Linux server the steps are different.

  1. Install the require NFS packages:
    apt install nfs-kernel-server
  2. Create a directory, we’ll call it “nfs-export” on your root fs on the server:
    mkdir /nfs-export/
  3. Then create a directory for the Raspberry Pi NFS Root:
    mkdir /nfs-export/PI-ImageName
  4. Now edit your /etc/exports file and add this line to the file to export the path:
    /nfs-export/PI-ImageName     IPorNetworkRange(rw,no_root_squash,async,insecure)
  5. Reload the NFS exports to take affect:
    exportfs -ra

Take note of the mount point and/or NFS export path, as this is the directory your Raspberry Pi will need to mount to access it’s NFS root. This is also the directory you will be copying your SD Card Linux install root FS to.

Copy the Linux install to the NFS Export

When you’re ready to copy your SD Card Linux install to your NFS Export, you’ll need to do the following. In my case I’ll be using an Ubuntu desktop computer to perform these steps.

When I insert the SD Card containing the Raspberry Pi Linux image, it appeared as /dev/sdb on my system. Please make sure you are using your proper device names to avoid using the wrong one to avoid writing or using the wrong disk.

Instructions to copy the root fs from the SD card to the NFS root export:

  1. Mount the root partition of the SD Card Linux install to a directory. In my case I used directory called “old”.
    mount /dev/sdb2 old/
  2. Mount the NFS Export for the NFS Root to a directory. In my case I used a directory called “nfs”.
    mount IPADDRESS:/nfs-export/PI-ImageName nfs/
  3. Use the rsync command to transfer the SD card Linux install to the NFS Root Export.
    rsync -avxHAXS --numeric-ids --info=progress2 --progress old/ nfs/
  4. Unmount the directories.
    umount old/
    umount nfs/

Once this is complete, your OS root is now copied to the NFS root.

Copy and Modify the boot SD Card to use NFS Root

First we have to copy the boot partition from the SD Card Linux install to the boot SD card, then we need to modify the contents of the new boot SD card.

Top copy the boot files, follow these instructions.

  1. Mount the boot partition of the SD Card Linux install to a directory. In my case I used directory called “old”.
    mount /dev/sdb1 old/
  2. Mount the new boot partition of the boot SD card to a new directory. In my case I used the directory called “new”.
    mount /dev/sdc1 new/
  3. Use the rsync command to transfer the SD card Linux install boot partition to the new boot SD card.
    rsync -avxHAXS --numeric-ids --info=progress2 --progress old/ new/
  4. Unmount the directories.
    umount old/
    umount new/

Now there are few steps we have to take to make to the boot SD card boot to an NFS Root.

We have to make a modification to the PI boot command. It is different depending on which Linux image (Ubuntu or Raspbian) you’re using.

First, insert the boot SD card, and mount it to a temporary directory.

mount /dev/sdc1 new/

If you’re running Ubuntu, your existing nobtcmd.txt should look like this:

dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait

We’ll modify and replace some text to make it look like this. Don’t forget to change the command to reflect your IP and directory:

dwc_otg.lpm_enable=0 console=tty1 root=/dev/nfs nfsroot=IPADDRESS:/nfs-export/PI-Ubuntu,tcp,rw ip=dhcp rootfstype=nfs elevator=deadline rootwait

For Raspbian, your existing cmdline.txt should look like this:

console=serial0,115200 console=tty1 root=PARTUUID=97709164-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait

We’ll modify and replace some text to make it look like this. Don’t forget to change the command to reflect your IP and directory:

console=serial0,115200 console=tty1 root=/dev/nfs nfsroot=IPADDRESS:/nfs-export/PI-Raspbian,tcp rw vers=3 ip=dhcp rootfstype=nfs elevator=deadline rootwait

Once you make the modifications, save the file and unmount the SD card.

Your SD card is now ready to boot.

Boot using SD Card and test NFS Root

At this point, insert the boot SD Card in your Raspberry Pi and attempt to boot. All should be working now and it should boot and use the NFS root!

If you’re having issues, if the boot process stalls, or something doesn’t work right, look back and confirm you followed all the steps above properly.

You’re done!

You’re now complete and have a fully working NFS root for your Raspberry Pi. You’ll no longer worry about storage, have high speed access to it, and you’ll have some new skills!

And don’t forget to check out these Handy Tips, Tricks, and Commands for the Raspberry Pi 4!

Mar 212020
CanaKit Raspberry Pi 4 Case with cables

During a previous project I needed to create a fresh and clean boot partition for a Raspberry Pi. I needed to create the partition layout required for the Raspberry Pi to see and boot a Linux kernel from.

There are many guides on the internet on how to write a Raspberry Pi image (which includes the system-boot partition), but I wanted a clean and fresh partition layout, without the additional partitions containing the Linux operating system.

I was creating a new Micro SD card with the purpose of using an NFS Root for the Raspberry Pi. For those of you that don’t know, you can boot a Raspberry Pi (or Linux computer) from local media, whether it’s a CD, USB Stick, Micro SD, or hard drive, and then have the actual operating system root file system be loaded via NFS. You can also use PXE to boot the kernel requiring no local storage, but that’s beyond the scope of this article.

Raspberry Pi default Partition layout

Below, we’ll look at the default partition layout you’d see on a Raspberry Pi using a prebuild linux image.

Disk /dev/sda: 59.6 GiB, 64021856256 bytes, 125042688 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x97709164
Device     Boot  Start       End   Sectors  Size Id Type
/dev/sda1         8192    532479    524288  256M  c W95 FAT32 (LBA)
/dev/sda2       532480 125042687 124510208 59.4G 83 Linux

I’m using a USB to Micro SD adapter to view the partitions on this card, so it’s being presented to the system as “/dev/sda”. On a normal computer “/dev/sda” is the first hard drive (usually the OS) so be careful when using these commands.

You’ll notice that “/dev/sda1” is the Raspberry Pi boot partition, with an Id of 3, and has the type of “W95 FAT32 (LBA)”.

The second partition which is the filesystem root (which I moved to NFS), is “/dev/sda2”, with an Id of 83, and has a type of “Linux”.

Creating a fresh partition layout with only the boot partition

In this guide we’re going to setup a Micro SD card with a fresh boot partition for the Raspberry Pi from scratch. We are not using an image and we are not using the expansion feature.

We’re going to assume that your destination SD card is empty. If it isn’t, you’ll need to delete all the partitions using “fdisk /dev/device”, and then deleted them with “d”.

Alternatively, to delete existing partition information you can wipe the MBR and partition table with the following command. Replace “/dev/device” with the actual device label for the card. Note that this will render existing data useless and unrecoverable.

dd if=/dev/zero of=/dev/DEVICE bs=512 count=1

Please Note: Make sure you are running this command on the right device. Afterwards, unplug and re-insert the SD card.

Creating the layout

On an empty Micro SD card:

  1. Open fdisk on your card.
    fdisk /dev/sda
  2. Press “n” to create a partition.
  3. Press “p” to make it a primary partition.
  4. Press “1” to make it the first partition in the table.
  5. Press <enter> to accept the default on start sector.
  6. Type +size to choose the size. In my case I want 1GB, so I’ll type “+1G”.
  7. After it’s created, press “a” to make it bootable.
  8. Now we press “p” to print and view the partition table, as shown below.
    Command (m for help): p
    Disk /dev/sda: 3.7 GiB, 3965190144 bytes, 7744512 sectors
    Geometry: 122 heads, 62 sectors/track, 1023 cylinders
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: dos
    Disk identifier: 0x4eb27b84
    Device Boot Start End Sectors Size Id Type
    /dev/sda1 * 2048 2099199 2097152 1G 83 Linux
  9. No we need to set the partition type. Press “t” to set a partition type, choose the partition, and type “c” for “W95 FAT32 (LBA)”.
  10. We’re now left with this partition table.
    Image of a new clean Raspberry Pi Boot Partition Layout
  11. Press “w” to write and save, and exit fdisk.
  12. We now need to format the partition. Run the following command on your device.
    mkfs.vfat /dev/sda1

Finally, you can now set a label to the partition. Ubuntu uses the label “system-boot” whereas Raspbian uses “boot”. You can set it with the following command:

fatlabel /dev/device NEW_LABEL

You now have a clean partition layout that can be used to boot a Raspberry Pi. Remember that this is just the partition layout and the files are still needed from an image or your current running instance. These can simply be copied over.

In my case, I just mounted an old and the new partitions to directories and copied the data over. This allowed me to modify the new boot partition and ultimately make it boot in to an NFS root.

If you need just a simple boot partition, you don’t need to purchase large Micro SD cards.

Mar 212020
CanaKit Raspberry Pi 4 case open with Fan Kit and running

In this post you’ll find a list of handy tips, tricks, and commands for your new Raspberry Pi 4.

I’ve been maintaining a document to record these so I can search and re-use them, and figured I’d share them on the blog for others to use as well.

I’m hoping to target both Raspbian and Ubuntu Server for the Raspberry Pi 4. If you have any feedback or input, please leave a comment!

Enable 64-Bit Kernel on Raspbian

Enables 64-bit kernel on Raspbian, however remember that the userspace is still 32-bit.

  1. Run “rpi-update” to make sure you’re running latest firmware and kernel.
  2. Add “arm_64bit=1” to “/boot/config.txt”
    sudo echo arm_64bit=1 >> /boot/config.txt
  3. Restart

Remove, comment out, or set the value to 0 to go back to a 32-bit kernel.

Get CPU Temperature on Raspbian

Run the command “sudo vcgencmd measure_temp” to get the CPU temperature on Raspbian.

[email protected]:~ $ sudo vcgencmd measure_temp

Get CPU Temperature on Ubuntu Server

Run the command “paste <(cat /sys/class/thermal/thermal_zone/type) <(cat /sys/class/thermal/thermal_zone/temp) | column -s $’\t’ -t | sed ‘s/(.)..$/.\1°C/'” as root to get the CPU temperature on Ubuntu Server.

[email protected]:~# paste <(cat /sys/class/thermal/thermal_zone/type) <(cat /sys/class/thermal/thermal_zone/temp) | column -s $'\t' -t | sed 's/(.)..$/.\1°C/'
cpu-thermal  45.2°C

Add root CA (Certification Authority) certificates to the trust on Raspbian and Ubuntu Server

To add a root CA to your CA trust on your Linux instance, perform the following:

  1. Save your certificate as a friendly-filename.crt (CRT extension is important) and copy to “/usr/local/share/ca-certificates/”.
  2. Run the “update-ca-certificates” command as root or sudo.

Install Cockpit Remote Web Administration

To install cockpit on your Raspberry Pi, run the following command as root or sudo:

apt install cockpit

After this, login to Cockpit on your Pi by pointring your browser to https://PI-IP-ADDRESS:9090

To install the Target CLI to configure the iSCSI Target

In order to configure the Linux kernel iSCSI target, you need the “targetcli” application/binary.

To install on Raspbian, run the command as root or sudo:

apt install targetcli

To install on Ubuntu Server, run the command as root or sudo:

apt install targetcli-fb

Mar 182020
Raspberry Pi iSCSI Target with external USB drive attached

The Raspberry Pi 4 is a super neat little device that has a whole bunch of uses, and if there isn’t for something you’re looking for you can make one! As they come out with newer and newer generations of the Raspberry Pi, the hardware gets better, faster, and the capabilities greatly improve.

I decided it was time with the newer and powerful Raspberry Pi 4, to try and turn it in to an iSCSI SAN! Yes, you heard that right!

With the powerful quad core processor, mighty 4GB of RAM, and USB 3.0 ports, there’s no reason why this device couldn’t act as a SAN (in the literal sense). You could even use mdadm and configure it as a SAN that performs RAID across multiple drives.

Picture of a Raspberry Pi 4 with External USB 3 HD setup as an iSCSI Target and SAN
Raspberry Pi 4 with External USB 3 HD

In this article, I’m going to explain what, why, and how to (with full instructions) configure your Raspberry Pi 4 as an iSCSI SAN, an iSCSI Target.

Please Note: these instructions also apply to standard Linux PCs and Servers as well, but I’m putting emphasis that you can do this on SBCs like the Raspberry Pi.

A little history…

Over the years on the blog, I’ve written numerous posts pertaining to virtualization, iSCSI, storage, and other topics because of my work in IT. On the side as a hobby I’ve also done a lot of work with SBC (Single Board Computers) and storage.

Some of the most popular posts, while extremely old are:

You’ll notice I put a lot of effort specifically in to “Lio-Target”…

When deploying or using Virtualization workloads and using shared iSCSI storage, the iSCSI Target must support something called SPC-3/SPC-4 Reservations.

SPC-3 and SPC-4 reservations allow a host to set a “SCSI reservation” and reserve the blocks on the storage it’s working with. By reserving the storage blocks, this allows numerous hosts to share the storage. Ultimately this is what allows you to have multiple hosts accessing the same volume. Please keep in mind both the iSCSI Target and the filesystem must support clustered filesystems and multiple hosts.

Originally, most of the open source iSCSI targets including the one that was built in to the Linux kernel did not support SCSI reservations. This resulted in volume and disk corruption when someone deployed a target and connected with multiple hosts.

Lio-Target specifically supported these reservations and this is why it had my focus. Deploying a Lio-target iSCSI target fully worked when using with VMware vSphere and VMware ESXi.

Ultimately, on January 15th, 2011 the iSCSI target in the Linux kernel 2.6.38 was replaced with Lio-target. All new Linux kernels use the Lio-Target as it’s iSCSI target.

What is an iSCSI Target?

An iSCSI target is a target that contains LUNs that you connect to with an iSCSI initiator.

The Target is the server, and the client is the initiator. Once connected to a target, you can directly access volumes and LUNs using iSCSI (SCSI over Internet).

What is it used for?

iSCSI is mostly used as shared storage for virtual environments like VMware vSphere (and VMware ESXi), as well as Hyper-V, and other hypervisors.

It can also be used for containers, file storage, remote access to drives, etc…

Why would I use or need this on the Raspberry Pi 4?

Some users are turning their Raspberry Pi’s in to NAS devices, whynot turn it in to a SAN?

With the powerful processor, 4GB of RAM, and USB 3.0 ports (for external storage), this is a perfect platform to act as a testbed or homelab for shared storage.

For virtual environments, if you wanted to learn about shared storage you could deploy the Raspberry Pi iSCSI target and connect to it with one or more ESXi hosts.

Or you could use this to remotely connect to a disk on a direct block level, although I’d highly recommend doing this over a VPN.

How do you connect to an iSCSI Target?

As mentioned above, you normally connect to an iSCSI Target and volume or LUN using an iSCSI initiator.

Using VMware ESXi, you’d most likely use the “iSCSI Software Adapter” under storage adapters. To use this you must first enable and configure it under the Host -> Configure -> Storage Adapters.

Image of the iSCSI Initiator software adapter configuration on VMware vSphere of an ESXi host.
VMware vSphere Host iSCSI Initiator Software Adapter

Using Windows 10, you could use the iSCSI initiator app. To use this simply search for “iSCSI Initiator” in your search bar, or open it from “Administrative Tools” under the “Control Panel”.

The Windows 10 iSCSI Initiator (iSCSI Properties) window.
Windows 10 iSCSI Initiator (iSCSI Properties)

There is also a Linux iSCSI initiator that you can use if you want to connect from a Linux host.

What’s needed to get started?

To get started using this guide, you’ll need the following:

  • Raspberry Pi 4
  • Ubuntu Server for Raspberry Pi or Raspbian
  • USB Storage (External HD, USB Stick, preferably USB 3.0 for speed)
  • A client device to connect (ESXi, Windows, or Linux)
  • Networking gear between the Raspberry Pi target and the device acting as the initiator

Using this guide, we’re assuming that you have already installed, are using, and have configured linux on the Raspberry Pi (setup accounts, and configured networking).

The Ubuntu Server image for Raspberry Pi comes ready to go out of the box as the kernel includes modules for the iSCSI Target pre-built. This is the easier way to set it up.

These instructions can also apply to Raspbian Linux for Raspberry Pi, however Raspbian doesn’t include the kernel modules pre-built for the iSCSI target and there are minor name differences in the apps. This is more complex and requires additional steps (including a custom kernel to be built).

Let’s get started, here’s the instructions…

If you’re running Raspbian, you need to compile a custom kernel and build the iSCSI Target Core Modules. Please follow my instructions (click here) to compile a custom kernel on Raspbian or Raspberry Pi. When you’re following my custom kernel build guide, in addition after running “make menuconfig”:

  1. Navigate to “Device Drivers”.
  2. Select (using space bar) “Generic Target Core Mod (TCM) and ConfigFS Infrastructure” so that it has an <M> (for module) next to it. Then press enter to open it. Example below.
    <M> Generic Target Core Mod (TCM) and ConfigFS Infrastructure
  3. Select all the options as <M> so that they compile as a kernel module, as shown below.
     --- Generic Target Core Mod (TCM) and ConfigFS Infrastructure
    <M> TCM/IBLOCK Subsystem Plugin for Linux/BLOCK
    <M> TCM/FILEIO Subsystem Plugin for Linux/VFS
    <M> TCM/pSCSI Subsystem Plugin for Linux/SCSI
    <M> TCM/USER Subsystem Plugin for Linux
    <M> TCM Virtual SAS target and Linux/SCSI LDD Fabcric loopback module
    <M> Linux-iSCSI.org iSCSI Target Mode Stack
  4. Save the kernel config and continue following the “compile a custom raspberry pi kernel” guide steps.

If you’re running Ubuntu Server, the Linux kernel was already built with these modules so the action above is not needed.

We’re going to assume that the USB drive or USB stick you’ve installed is available on the system as “/dev/sda” for the purposes of this guide. Also please note that when using the create commands in the entries below, it will create it’s own unique identifiers on your system different from mine, please adjust your commands accordingly.

Let’s start configuring the Raspberry Pi iSCSI Target!

  1. First we need to install the targetcli interface to configure the target.
    As root (or use sudo) run the following command if you’re running Ubuntu Server.
    apt install targetcli-fb
    As root (or use sudo) run the following command if you’re running Raspbian.
    apt install targetcli
  2. As root (or using sudo) run “targetcli”.
    Running the targetcli command
  3. Create an iSCSI Target and Target Port Group (TPG).
    cd iscsi/
    Command to create a TPG iSCSI Target
  4. Create a backstore (the physical storage attached to the Raspberry Pi).
    cd /backstores/block
    create block0 /dev/sda
    Creating an iSCSI Target Backstore command
  5. Create an Access Control List (ACL) for security and access to the Target.
    cd /iscsi/iqn.2003-01.org.linux-iscsi.ubuntu.aarch64:sn.eadcca96319d/tpg1/acls
    create iqn.1991-05.com.microsoft:your.iscsi.initiator.iqn.com
    Creating an ACL inside of targetcli for the iSCSI Target
  6. Add, map, and assign the backstore (block storage) to the iSCSI Target LUN and ACL.
    cd /iscsi/iqn.2003-01.org.linux-iscsi.ubuntu.aarch64:sn.eadcca96319d/tpg1/luns
    create /backstores/block/block0
    Mapping a backstore to LUN and ACL in TargetCLI
  7. Review your configuration.
    cd /
    Reviewing the configuration in TargetCLI
  8. Save your configuration and exit.
    Saving the configuration and exiting the targetcli interface

That’s it, you can now connect to the iSCSI target via an iSCSI initiator on another machine.

For a quick example of how to connect, please see below.

Connect the ESXi Initiator

To connect to the new iSCSI Target on your Raspberry Pi, open up the configuration for your iSCSI Software Initiator on ESXi, go to the targets tab, and add a new iSCSI Target Server to your Dynamic Discovery list.

Add iSCSI Server to the Dynamic Discovery list on the iSCSI Software Initiator on ESXi
ESXi adding iSCSI Target Server (SAN) to iSCSI Software Initiator Dynamic Discovery

Once you do this, rescan your HBAs and the disk will now be available to your ESXi instance.

Connect the Windows iSCSI Initiator

To connect to the new iSCSI Target on Windows, open the iSCSI Initiator app, go to the “Discovery” tab, and click on the “Discover Portal” button.

Adding an iSCSI Target Server to the Windows iSCSI Software Initiator
Add iSCSI Target Server to Windows iSCSI Initiator

In the new window, add the IP address of the iSCSI Target (your Raspberry Pi), and hit ok, then apply.

Now on the “Targets” tab, you’ll see an entry for the discovered target. Select it, and hit “Connect”.

The targets list on Windows iSCSI Software Initiator
Windows iSCSI Initiator Targets List

You’re now connected! The disk will show up in “Disk Management” and you can now format it and use it!

Here’s what an active connection looks like.

The Microsoft iSCSI Initiator window open, showing an active connection to an iSCSI target, and iSCSI disk
Windows 10 iSCSI Initiator connect to iSCSI Target presenting a disk

That’s all folks!


There you have it, you now have a beautiful little Raspberry Pi 4 acting as a SAN and iSCSI Target providing LUNs and volumes to your network!

Picture of Raspberry Pi 4 iSCSI Target running Ubuntu Server with External USB 3 HD
Raspberry Pi 4 iSCSI Target with External USB 3 HD

Leave a comment and let me know how you made out or if you have any questions!

Mar 172020
Picture of Raspberry Pi 4 box and Raspberry Pi 4 board below box

So you’ve got a shiny new Raspberry Pi 4 and you need to compile a fresh and custom Linux kernel on Raspbian. You might need some features, some kernel modules, or you just want to compile the latest version from source.

I’m doing various projects (and blog posts) and with one of the projects, I found I needed to compile and enable a kernel module that wasn’t built in to the latest Raspbian image for the Pi 4.

This guide is also great if you just want to learn how to compile the kernel yourself!


You may find that this guide is slightly different that the guide on the Raspberry Pi website and other sites. I like to append a unique name to the kernel version so I don’t have to touch the existing kernels. This allows me to revert or run multiple different custom kernels and switch back and forth.

Please note: You must be using a 32-bit kernel (or the default Raspbian kernel) to compile a new 32-bit kernel. You will not be able to compile a new kernel (32-bit or 64-bit) if you have booted in to the 64-bit kernel using the “arm_64bit=1” switch in “config.txt”. I’ve tried to compile a 64-bit kernel on Raspbian, but have not yet been able to do so. I’ll update with a new post once I figure it out.

And don’t forget, this can take some time and is CPU intensive. I installed a fan to help cool the temperatures while compilling!

This guide will compile a 32-bit kernel.

  1. Install some packages required to building and compiling.
    apt install raspberrypi-kernel-headers build-essential bc git wget bison flex libssl-dev make libncurses-dev
  2. Create a directory for us to work in.
    mkdir kernel
    cd kernel
  3. Clone the latest kernel sources using GIT.
    git clone --depth=1 https://github.com/raspberrypi/linux
  4. Setup the kernel configuration for compiling.
    cd linux
    make bcm2711_defconfig
  5. Make any changes you want to the kernel configuration and append a friendly local version name by using make menuconfig.
    make menuconfig

    To change the friendly name, navigate to “General Setup” and select/modify “Local Version – append to kernel release”.
    (-v7lstephen) Local version - append to kernel release
  6. Compile the kernel, modules, and device tree blobs.
    make -j4 zImage modules dtbs
  7. Install compiled modules.
    make modules_install
  8. Copy the kernel, modules, and other files to the boot filesystem.
    cp arch/arm/boot/dts/*.dtb /boot/
    cp arch/arm/boot/dts/overlays/*.dtb* /boot/overlays/
    cp arch/arm/boot/dts/overlays/README /boot/overlays/
    cp arch/arm/boot/zImage /boot/kernel-stephen.img
  9. Configure the PI to boot using the new kernel by modifying and adding the below line to “/boot/config.txt”.
  10. Reboot!

Bam! You’re now using your shiny new Linux kernel on the Raspberry Pi 4!

To rescue a failed build or if the Pi won’t boot

If for some reason the Pi won’t boot, you can recover the previous kernel since we used a new name with the new kernel.

To rescue the image you’ll need another Linux computer that can read the Micro-SD card.

  1. Insert the Micro-SD Card in the computer.
  2. Mount the /boot/ filesystem on the Micro SD card to a local directory.
  3. Edit the “config.txt” file and remove the “kernel=kernel-name.img” line we made above, or alternatively comment it out by inserting a “#” before the line.
  4. Save the file.
  5. Unmount the partition.
  6. Insert in the Raspberry Pi and boot!

You should now be back up and running and should be able to try again!

Leave some feedback and let me know if it worked for you. In the future I’ll be doing another post on compiling a 64-bit kernel for the Raspberry Pi 4 on Raspbian.

Mar 162020
Picture of Raspberry Pi 4 box and Raspberry Pi 4 board below box

So you just loaded up Ubuntu Server on your Raspberry Pi 4 using the latest Ubuntu Server Pi image and when you try logging in with the default username and password of ubuntu:ubuntu, you get the error “Authentication token Manipulation error” when you try to change the default password and log in.

This occurs on a fresh image write to an SD card using the Ubuntu 18.04.4 LTS 64-bit image. This may occur on other images and other versions of Ubuntu and other versions of the Raspberry Pi.

The filename for the image with the issue is:


How to Fix

After doing some research, I found out that there was an issue with a password file or the PAM database on the image. I figured that it was best to try to log in first using the default credentials, and then we can worry about changing the password later.

To do this I decided to modify the “cloud-init” scripts. I mounted the SD-Card on another Linux system, opened the “/boot/firmware/user-data”, and changed the “expire” setting on the ubuntu user from true to false.

The Instructions

  1. Mount the SD-Card on another Linux system.
  2. Navigate to the boot filesystem, and then open the “user-data” file inside of the firmware directory using nano or vi. The full path on the SD-Card is:
  3. Scroll down to this section.
      expire: true
      - ubuntu:ubuntu

    Change the “expire: true” to “expire: false”.
  4. Exit the file and save.
  5. Properly unmount the SD-Card (using umount).
  6. Boot up the Pi with the Micro-SD card.

You should now be able to log in using the username ubuntu and password ubuntu without being asked to change your password, and without seeing the error.

Once you have logged in, change the password to this account by using “passwd”.

Hope it helps!

Jun 302012

As most of you know, I have 2 Raspberry Pi.  One has tons of storage, that I do a lot of my hacking on, and development. The other has less storage, I usually keep clean, and don’t do anything funky on it. I like the second to have a clean usable image (even though this doesn’t make sense, and I don’t use it in production, I call the second my “Production Pi”, the first is my “devpi”).

For my development Raspberry Pi, I maintain two installs. One install is a Fedora 17 ARM install with my own kernel that all fits on a 16GB SD card, but my second install (the crazy packed install) actually boots it’s kernel off a small 2GB SD card, and loads it’s rootfs off of a external 500GB USB drive. The SD card install was easy, however the rootfs on USB took a bit of work, I’m going to share with you how I did this.

To have your Raspberry Pi use a rootfs off a USB drive you will need to know what modules your USB drive currently uses on a un-modified Raspberry Pi kernel, and you will need a seperate linux box, and SD card reader/writer to prepare the rootfs and perform these instructions. In my case, I used my 16GB install as a template and cloned it to my 500GB USB drive.

DISCLAIMER: These instructions are what I did to setup a rootfs on a USB device. I am in no way telling you to follow these instructions, and if you do, and any damage occurs I am not liable or responsable. These instructions if not followed properly can cause damage to your existing linux install on your computer and your Raspberry Pi. Only follow these if you understand them.


Let’s get started, first let’s prepare the new bootable SD card which loads our kernel:

1) First we want to take a image of the partition table, and boot partition we already boot from on our existing working Raspberry Pi Fedora 17 ARM install. We only want to image the partition table and boot partition. First we use fdisk to find out how big the boot partition is:

fdisk /dev/sdb

Press “p” and hit enter to print the partition table. In my case it outputted:

Welcome to fdisk (util-linux 2.21.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): p

Disk /dev/sdb: 16.0 GB, 16009658368 bytes
64 heads, 32 sectors/track, 15268 cylinders, total 31268864 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *          63     1044224      522081    c  W95 FAT32 (LBA)
/dev/sdb2         1044225    27074558    13015167   83  Linux
/dev/sdb3        27074559    31268863     2097152+  83  Linux

This shows that my boot partition starts at sector 63 and goes till 1044224. Each sector is 512 bytes, for a total boot partition that is 534642688 bytes, or ~509 megabytes. The reason that the partition starts at sector 63, is because everything prior to that on the SD card contains partition information, etc… When we image, we will start from the beginning to grab that partition information. After you record the End sector for your boot partition, hit “q” and enter to exit fdisk.

Let’s create the image. In order to do this, you need to find out what sd* device your SD card is on your current system. If you do a dmesg after connecting the sd card, you should see it. Make sure you choose the correct sd* or you could damage your current linux system. In my case, after I connected the SD card to my linux workstation, it became /dev/sdb.

Let’s make the image:

dd if=/dev/sdb of=image.img bs=512 count=1044224

This will create an image called image.img of the boot partition and partition table. Notice how we use a bs=512 (this is because each sector is 512 bytes), and a count=1044224 (this is the end of the boot partition).


2) Let’s copy the SD Card’s rootfs to a external drive! Hook up your empty external hard drive. On this drive we will do a few things. First, partition the drive so we have two partitions (one for the rootfs, and one for swap), then we will make the filesystems, and finally copy over the rootfs to the new drive.

In my case, when I hooked up my external drive to my computer, it was assigned as /dev/sdc, the SD card is still /dev/sdb. Keep in mind this may be different on your computer, if you use the incorrect values, you may actually damage your current linux workstation. You can always use dmesg after hooking up a usb device to find out what sd* it was assigned.

First let’s create a patition table on the usb drive. Again, on my computer the usb drive is sdc, your’s may be different.

fdisk /dev/sdc

In fdisk, we will create a partition for an ext4 filesystem. Press “n” for new partitions, follow the instructions, create a primary partition, and let fdisk choose the start and ending. This will use all the space on the hard drive. When done creating, press “w” and enter to write the partition table. This created the /dev/sdc1 partition. Now we need to create the filesytem and label it.

mkfs.ext4 /dev/sdc1 -L rootfs

This creates a ext4 filesystem and labels it “rootfs”.


3) Let’s copy the old rootfs to the new one. First we need to mount both of them, and copy everything over.

To do this, we are going to create two folders which we will mount both of the rootfs too.

mkdir rootsource

mkdir rootdest

Then we will mount them:

mount /dev/sdb2 rootsource/

mount /dev/sdc1 rootdest/

And finally, we will copy over the rootfs and then unmount the mounts

cp -afv rootsource/* rootdest/

umount rootsource/

umount rootdest/

You have now copied over the root filesystem from your SD card, to the external USB drive.


4) Now we are done with the original SD card, you can remove it, and put in your new SD card which will be used to boot the kernel.

First we need to write the image that contains the partition table, and boot partition. We do this by writing the image file you created above to the NEW SD card. Again, MAKE SURE YOU KNOW WHAT /dev/sd* device the new card is being recognized as.

dd if=image.img of=/dev/sdb

This will write the image to your new SD card. After this, even though we only took the partition table, and the boot partition, the partition table still contains information of the old rootfs. Let’s clean this up (even though we don’t have to).

fdisk /dev/sdb

Inside of fdisk, we will hit “D” to delete all bogus partitions EXCEPT for the boot partition. In my case I had 2 I had to remove, partition 2 and 3. You will probably only have one, which will be partition 2. After you’re done deleting, hit “w” and enter to save.


4) Now we have to go to the boot partition and prepare it to boot off the USB drive.

Let’s create a directory to mount and work inside of, then we will update the cmdline.txt file which the Raspberry Pi uses for information to boot from.

mkdir bootmount

mount /dev/sdb1 bootmount

nano bootmount/cmdline.txt

We are now looking at the cmdline.txt file. We need to update it to boot from the USB drive rootfs partition which we created above. Keep in mind, when you hook the USB drive up to your Raspberry Pi, it will be the only /dev/sd* device, so it will probably appear as /dev/sda if you have no other USB drives connected to it. Update yours, here’s an example of mine:

dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/sda1 rootfstype=ext4 rootwait text

Save and close the cmdline.txt and then unmount.

umount bootmount


So here is where we sit (in order):

-We imaged the old boot partition

-We prepared the new USB drive

-We copied the rootfs to the new USB drive from the SD card

-We wrote the boot image to the new SD card

-We configured the new SD card to boot using the external drive as a root fs.


All we have left to do is to is connect the USB drive to your Raspberry Pi, and insert the new boot SD card in to the Pi as well. Boot your Raspberry Pi, if all is well, it should boot off your new rootfs (on the USB drive). This will improve speed since the USB drive is WAY faster than the SD card, and now you’re SD card will only be used to boot the kernel.

Important Notes:

-If these instructions don’t work, chances are your USB device may require a driver that is not built in your current Raspberry Pi linux kernel, you will need to identify what module your USB device uses by issuing a “lsmod” when you have booted off your old SD card, then re-compiling the kernel with this built in to the kernel and NOT as a module. Instructions on compiling a kernel can be found here: http://www.stephenwagner.com/?p=616

-If you really feel comfortable with Linux, after you do this, you could re-size the partitions, and create a SWAP partition for your Raspberry Pi. ONLY setup a Swap partition if you are using an external USB drive, as SD Card’s are NOT fast enough, and swap’ing can actually limit the life of a SD card due to the I/Os.

Jun 302012

A number of projects I’m currently working on and some that I haven’t even started yet, require me to compile my own kernel. The reason I need to do this is to be able to build various code in to a live running kernel, or make various modules available for hardware/applications I need or will need in the future.


A few reasons why you may require a custom built kernel:

-Hardware support for particular hardware you are using

-Support for USB drives, USB devices, Wifi, etc…

-Support for kernel features (special networking features, software features, etc…)

-Built support for booting off different rootfs devices (NFS, iSCSI, USB devices)

The list goes on and on, but you have the idea. And chances are if you came to this page, you already have your reason, you’re just trying to figure out how. Well it’s pretty easy on Fedora 17 ARM on your Raspberry Pi. Keep in mind you can use a cross-compiler and build the kernel on your desktop x86 machine and move it over, but I preferred to actually build the kernel on my Raspberry Pi.

Here’s how:

1) First let’s grab all the packages we need. We will issue a yum command to make sure we have the compilers, libraries, and other stuff we will need. We will also install “screen” so that we can hide the terminal session where we are compiling so you don’t need to leave your ssh session open.

yum install gcc gcc-c++ gcc-gfortran glib libtool gtk+ gtk+extra gtk2 git ncurses-devel kernel-headers screen

This will install everything and any dependencies that are required.


2) Let’s start a screen session. A screen session is a terminal session you can attach/detach on demand. Think of it as a terminal that you can keep open even in the back ground. You can start it in a ssh session, close your ssh session, and then connect back to the screen session later when you ssh back in.

screen -S kernelcompile

This starts a session called kernelcompile. At any time you can detach the session by pressing Ctrl+A then pressing the “D” button. To re-attach, simply issue “screen -r”.


3) Download the kernel sources (with the Raspberry Pi patches built in). You can grab a copy off of the snapshot off the github Raspberry Pi kernel repo.

wget https://github.com/raspberrypi/linux/tarball/rpi-patches
mv rpi-patches kernel.tar.gz

tar zxvf kernel.tar.gz

This will download the sources as a file called rpi-patches, and we just rename it then untar it.


4) In my case, it resulted in a folder called “raspberrypi-linux-f679f05”. Here we change directory in to the kernel source, and setup the default kernel config options. And then run “make menuconfig” to bring up the script to allow us to configure our kernel. This is where you choose your options as to what you want to build in to the kernel, and what you want to build as kernel modules that can be loaded after the kernel is booted.

cd raspberrypi-linux-f679f05

cp arch/arm/configs/bcmrpi_defconfig .config

make menuconfig

IMPORTANT: You need to append a version on to the kernel version number, so that you don’t screw your current existing modules up (since there’s a chance if you didn’t edit this, they would be overwritten, or your build would fail when building your modules. You can do this in “make menuconfig” under General Setup. Or you can edit your .config file:

nano .config

Look for “CONFIG_LOCALVERSION”. In my case, mine is set to: CONFIG_LOCALVERSION=”.001″ this will put a .001 at the end of the kernels versions tring, and also after the module version number. Example: /lib/modules/

After you are done with menuconfig, simply exit out of “make menuconfig” and save your config.


5) Now it’s time to start the compiling process. Typing “make” will compile the kernel, and typing “make modules_install” will build the modules and install them to the “/lib/modules” directory.


make modules_install


If the building completed then you have successfully built a linux kernel for the Raspberry Pi and also built/installed the modules that come along with it… We aren’t done yet though. We issue a cd.. to get out of the kernel source directory and back in to the directory that holds it.


6) Next we need to download the Raspberry Pi tools which will contain a python application called “imagetool-uncompressed.py”.

wget https://github.com/raspberrypi/tools/zipball/master

mv master tools.zip

unzip tools.zip

We have downloaded the tools, and unzipped them. It might be an idea to issue a “ls” to note the name of the directory it created.


7) No we need to enter the tools directory. In my case the directory was called “raspberrypi-tools-772201f” but it will be different for you. After we will build the image.

cd raspberrypi-tools-772201f/

cd mkimage/

python2 imagetool-uncompressed.py /path/to/kernel/source/raspberrypi-linux-f679f05/arch/arm/boot/Image

Note, you will have to also change the directory after the imagetool command to directory where you built your kernel. Once this runs, you will be left with a kernel.img file inside of the directory you are currently in. First we will back up the existing kernel, then copy the new kernel to your /boot/ directory.

IMPORTANT: Once you overwrite your existing kernel, if your kernel build was bad, or you don’t have what’s needed to boot, your kernel MAY not boot. This will cause you to restore your old kernel which I am NOT going in to in this blog post!

mv /boot/kernel.img /boot/kernel.img.bak

cp kernel.img /boot/

Above, we have changed the name of your current kernel and added a .bak to the end. Then we copied the new kernel made to the boot directory.


Now you can simply reboot your Raspberry Pi, and BAM, with luck you’ll be running your own kernel.


PS. If you ever want to back this kernel up, all you need to do is copy kernel.img (in /boot/) to a safe place, and make a copy of your /lib/modules/kernelversion (in my case /lib/modules/ directory. If you ever re-image your Pi and want to use the kernel you just built, you can copy the kernel.img and the modules directory to the new image and you are good to go! Might also be worth while to make a backup of your kernel sources and imagetool if you need it again!

Jun 292012

As most of you have read, I received 2 X Raspberry Pi the other day. I’ve been actively hacking and working away on these lovely little devices.

An updated post on setting up a Raspberry Pi as an iSCSI Target can be found here!

One of the projects I wanted to do, was get Lio-Target (iSCSI Target) running on the Pi. I know that the Pi doesn’t have gigabit networking, but I thought this would still be an interesting proof of concept. Anyways, I got it running, and I have succesfully connected to a USB storage device which was configured as a iSCSI target on my Pi, from my Windows 7 workstation.

This is a brief overview, I will be providing instructions in detail at a later date. Here’s how I did it:

1) Download Fedora 17 for ARM (build for Raspberry Pi).

2) Put latest Firmware and Kernel from Raspberry Pi github repo on to the boot partition. Resize my 16GB card so I have boot, root, and a 2 GB swap.

3) Download snapshot of Raspberry Pi kernel sources. I built the iSCSI Target as modules (I also threw in some other stuff for future projects but it’s not important right now).

3) Install compilers, libraries, etc for kernel build process.

4) Compile kernel

5) Build Raspberry Pi kernel image using Raspberry Pi image tools on github repo, copy to boot.

6) Boot off new kernel

7) Install Target CLI from yum (this was a nice change from compiling on my own), and then build Lio-Utils (this isn’t mandatory, but I like Lio-utils).

8) Configure target, connect, test.

Here’s a copy/paste of proof I have it running!

[[email protected] lio-utils.git]# uname -a
Linux fedora-arm #1 PREEMPT Thu Jun 28 16:40:46 MDT 2012 armv6l armv6l armv6l GNU/Linux
[[email protected] lio-utils.git]# w
09:12:10 up 34 min,  5 users,  load average: 0.99, 1.51, 1.10
USER     TTY      FROM              [email protected]   IDLE   JCPU   PCPU WHAT
root     pts/0    host.digitall 31Dec69 22:38   2.27s  0.02s tail -f /var/log/messages
root     pts/1    host.digitall 31Dec69  0.00s  1.19s  0.05s w
root     pts/2    host.digitall 08:58   13:24  13.89s 13.04s top
root     pts/3    host.digitall 31Dec69  0.00s  0.00s   ?    –
[[email protected] lio-utils.git]# /etc/init.d/target status
[—————————] TCM/ConfigFS Status [—————————-]
\——> iblock_0
HBA Index: 1 plugin: iblock version: v4.1.0-rc1-ml
\——-> array0
Status: ACTIVATED  Execute/Left/Max Queue Depth: 0/128/128  SectorSize: 512  MaxSectors: 240
iBlock device: sdb  UDEV PATH: /dev/sdb
Major: 8 Minor: 16  CLAIMED: IBLOCK
udev_path: /dev/sdb

[—————————] LIO-Target Status [—————————-]
\——> iqn.2003-01.org.linux-iscsi.fedora-arm.armv6l:sn.4682cf8cdeec
\——-> tpgt_1  TargetAlias: LIO Target
TPG Network Portals:
\——-> IP-hidden:3260
TPG Logical Units:
\——-> lun_0/30b42bf9f5 -> target/core/iblock_0/array0

Target Engine Core ConfigFS Infrastructure v4.1.0-rc1-ml on Linux/armv6l on
RisingTide Systems Linux-iSCSI Target v4.1.0-rc1
[[email protected] lio-utils.git]# cat /proc/cpuinfo
Processor       : ARMv6-compatible processor rev 7 (v6l)
BogoMIPS        : 697.95
Features        : swp half thumb fastmult vfp edsp java tls
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xb76
CPU revision    : 7

Hardware        : BCM2708
Revision        : 0002
Serial          : 00000000c1ad6318
[[email protected] lio-utils.git]#

Jun 272012

Well, it’s official. I am now a happy owner of two Raspberry Pi computers.

I’m busy tinkering away but I just wanted to upload some pictures and videos. I will be creating a separate post outlining what I’m hoping to accomplish with these little devices, so stay tuned!

Scroll to the bottom of this post for videos!

Element 14 shipping package (origina Toronto, Ontario)RS Electronics Box containing Raspberry Pi and 4GB SD Card










Two Raspberry Pi's in their shipping boxes (Left Element14, Right RS Electronics)Two Raspberry Pi on their electro-static wrappers.











Raspberry Pi hooked upRaspberry Pi









Raspberry Pi in operationRaspberry Pi turned On











The workbench for Raspberry Pi development












And here is a few videos of the first couple boots:

First Boot (Text Boot Image)


Second Boot (Graphic XFCE Image)