Context

I am using the https://wiki.jenkins.io/display/JENKINS/Amazon+EC2+Plugin for jenkins which allows me to dynamically provision new cloud instances as build slaves in AWS EC2.

I am launching ami-d834aba1 (Amazon Linux 2017.09.1).

The plugin supports providing user-data and block device mapping too, currently I provide config like this after reading https://cloudinit.readthedocs.io/en/latest/

User Data

#cloud-config
repo_update: true
repo_upgrade: all
package_upgrade: true

bootcmd:
 - [ cloud-init-per, once, mkfs, -t, ext4, /dev/nvme1n1 ]

fs_setup:
 - cmd: mkfs -t %(filesystem)s -L %(label)s %(device)s
   label: jenkins
   filesystem: 'ext4'
   overwrite: false
   device: '/dev/nvme1n1'

mounts:
 - [ /dev/nvme1n1, /jenkins, "ext4", "defaults,nofail", "0", "2" ]

users:
 - default
 - name: jenkins
   homedir: /jenkins
   lock_passwd: true
   ssh_authorized_keys:
     - a-key

Block Device Mapping

/dev/sdd=:100:true:gp2::encrypted

Desired Behaviour

The instance would launch and would attach a new 100GB encrypted EBS volume which would be formatted as ext4 and mounted at /jenkins as the home directory of the jenkins user.

Observed Behaviour

The instance launches, the 100GB encrypted EBS volume is created and attached to the EC2 instance (shows as in use and attached in AWS console). However,

1) df -h doesn't show the filesystem.

2) cat /etc/fstab /dev/nvme1n1 /jenkins ext4 defaults,nofail,comment=cloudconfig 0 2 does show it

3) sudo file -s /dev/nvme1n1 /dev/nvme1n1: data shows the volume as data formatted rather than ext4

4) sudo mount-a fails due to the filesystem not being ext4.

Manual Hack

If i manually SSH to the machine after boot and run:

sudo mkfs -t ext4 /dev/nvme1n1
mke2fs 1.42.12 (29-Aug-2014)
Creating filesystem with 26214400 4k blocks and 6553600 inodes
Filesystem UUID: 7a434f7a-c048-4c3d-8098-b810e2ff8f84
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
    4096000, 7962624, 11239424, 20480000, 23887872

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done   

Then sudo mount -a it seems to mount the volume.

Questions

Is there any way to have the device formatted and mounted automatically? I tried with and without the

bootcmd: - [ cloud-init-per, once, mkfs, -t, ext4, /dev/nvme1n1 ]

Ideally it'd happen all before the user gets created since the home directory of the new user is going to be on this new mount.

If the instance is stopped and started/restarted I'd not want to ideally lose all data by the reformatting happening again on boot.

cloud-init on Amazon Linux does not support the fs_setup module. Hence, your disk is not formatted. Furthermore the home directory /jenkins is created for the user, and used as a mount point. This hides the home directory.

I would suggest:

bootcmd:
 - test -z "$(blkid /dev/nvme1n1)" && mkfs -t ext4 -L jenkins /dev/nvme1n1
 - mkdir -p /jenkins

mounts:
 - [ "/dev/nvme1n1", "/jenkins", "ext4", "defaults,nofail", "0", "2" ]

runcmd:
 - useradd -m -b /jenkins jenkins