amazon-ec2

Ubuntu AWS instance reachability check failed


I've recently added more storage to the /dev/sda (it was 8GB..made it to 150GB) by stopping the instance, detaching the volume and adding storage and reattaching and then starting the instance. I get the following output:

backend at /local/domain/0/backend/vbd/547/2049
314572800 sectors of 512 bytes
**************************
[H[J  Booting 'Ubuntu 14.04.2 LTS, kernel 3.13.0-57-generic'



root  (hd0)

 Filesystem type is ext2fs, using whole disk

kernel  /boot/vmlinuz-3.13.0-57-generic root=LABEL=cloudimg-rootfs ro console=h

vc0 

initrd  /boot/initrd.img-3.13.0-57-generic



close blk: backend at /local/domain/0/backend/vbd/547/2049
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 3.13.0-57-generic (buildd@brownie) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 (Ubuntu 3.13.0-57.95-generic 3.13.11-ckt21)
[    0.000000] Command line: root=LABEL=cloudimg-rootfs ro console=hvc0 
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Centaur CentaurHauls
[    0.000000] ACPI in unprivileged domain disabled
[    0.000000] e820: BIOS-provided physical RAM map:
[    0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable
[    0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved
[    0.000000] Xen: [mem 0x0000000000100000-0x00000003c07fffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI not present or invalid.
[    0.000000] No AGP bridge found
[    0.000000] e820: last_pfn = 0x3c0800 max_arch_pfn = 0x400000000
[    0.000000] e820: last_pfn = 0x100000 max_arch_pfn = 0x400000000
[    0.000000] Scanning 1 areas for low memory corruption
[    0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
[    0.000000] init_memory_mapping: [mem 0x3bfe00000-0x3bfffffff]
[    0.000000] init_memory_mapping: [mem 0x3bc000000-0x3bfdfffff]
[    0.000000] init_memory_mapping: [mem 0x380000000-0x3bbffffff]
[    0.000000] init_memory_mapping: [mem 0x00100000-0x37fffffff]
[    0.000000] init_memory_mapping: [mem 0x3c0000000-0x3c07fffff]
[    0.000000] RAMDISK: [mem 0x02404000-0x0341efff]
[    0.000000] NUMA turned off
[    0.000000] Faking a node at [mem 0x0000000000000000-0x00000003c07fffff]
[    0.000000] Initmem setup node 0 [mem 0x00000000-0x3c07fffff]
[    0.000000]   NODE_DATA [mem 0x3be218000-0x3be21cfff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x00001000-0x00ffffff]
[    0.000000]   DMA32    [mem 0x01000000-0xffffffff]
[    0.000000]   Normal   [mem 0x100000000-0x3c07fffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x00001000-0x0009ffff]
[    0.000000]   node   0: [mem 0x00100000-0x3c07fffff]
[    0.000000] SFI: Simple Firmware Interface v0.81 http://simplefirmware.org
[    0.000000] smpboot: Allowing 8 CPUs, 0 hotplug CPUs
[    0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000fffff]
[    0.000000] e820: cannot find a gap in the 32bit address range
[    0.000000] e820: PCI devices with unassigned 32bit BARs may break!
[    0.000000] e820: [mem 0x3c0900000-0x3c0cfffff] available for PCI devices
[    0.000000] Booting paravirtualized kernel on Xen
[    0.000000] Xen version: 4.2.amazon (preserve-AD)
[    0.000000] setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:8 nr_node_ids:1
[    0.000000] PERCPU: Embedded 27 pages/cpu @ffff8803bdc00000 s81536 r8192 d20864 u262144
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 3872618
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: root=LABEL=cloudimg-rootfs ro console=hvc0 
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Memory: 15384188K/15736444K available (7392K kernel code, 1146K rwdata, 3408K rodata, 1336K init, 1448K bss, 352256K reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU dyntick-idle grace-period acceleration is enabled.
[    0.000000]  RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=8.
[    0.000000]  Offload RCU callbacks from all CPUs
[    0.000000]  Offload RCU callbacks from CPUs: 0-7.
[    0.000000] NR_IRQS:16640 nr_irqs:336 16
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] console [hvc0] enabled
[    0.000000] allocated 63438848 bytes of page_cgroup
[    0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups
[    0.000000] installing Xen timer for CPU 0
[    0.000000] tsc: Detected 2793.364 MHz processor
[    0.004000] Calibrating delay loop (skipped), value calculated using timer frequency.. 5586.72 BogoMIPS (lpj=11173456)
[    0.004000] pid_max: default: 32768 minimum: 301
[    0.004000] Security Framework initialized
[    0.004000] AppArmor: AppArmor initialized
[    0.004000] Yama: becoming mindful.
[    0.004000] Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes)
[    0.005853] Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    0.007312] Mount-cache hash table entries: 32768 (order: 6, 262144 bytes)
[    0.007348] Mountpoint-cache hash table entries: 32768 (order: 6, 262144 bytes)
[    0.007643] Initializing cgroup subsys memory
[    0.007652] Initializing cgroup subsys devices
[    0.007655] Initializing cgroup subsys freezer
[    0.007659] Initializing cgroup subsys blkio
[    0.007662] Initializing cgroup subsys perf_event
[    0.007665] Initializing cgroup subsys hugetlb
[    0.007710] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
[    0.007710] ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8)
[    0.007717] CPU: Physical Processor ID: 1
[    0.007719] CPU: Processor Core ID: 0
[    0.008390] Last level iTLB entries: 4KB 512, 2MB 0, 4MB 0
[    0.008390] Last level dTLB entries: 4KB 512, 2MB 0, 4MB 0
[    0.008390] tlb_flushall_shift: 6
[    0.044032] ftrace: allocating 28585 entries in 112 pages
[    0.052071] cpu 0 spinlock event irq 17
[    0.062489] Performance Events: unsupported p6 CPU model 62 no PMU driver, software events only.
[    0.063938] NMI watchdog: disabled (cpu0): hardware events not enabled
[    0.064000] installing Xen timer for CPU 1
[    0.064000] cpu 1 spinlock event irq 24
[    0.064003] SMP alternatives: switching to SMP code
[    0.096909] installing Xen timer for CPU 2
[    0.096920] cpu 2 spinlock event irq 31
[    0.097846] installing Xen timer for CPU 3
[    0.097855] cpu 3 spinlock event irq 38
[    0.098755] installing Xen timer for CPU 4
[    0.098763] cpu 4 spinlock event irq 45
[    0.099706] installing Xen timer for CPU 5
[    0.099716] cpu 5 spinlock event irq 52
[    0.100573] installing Xen timer for CPU 6
[    0.100582] cpu 6 spinlock event irq 59
[    0.101487] installing Xen timer for CPU 7
[    0.101496] cpu 7 spinlock event irq 66
[    0.102328] x86: Booted up 1 node, 8 CPUs
[    0.102413] devtmpfs: initialized
[    0.105670] EVM: security.selinux
[    0.105673] EVM: security.SMACK64
[    0.105675] EVM: security.ima
[    0.105677] EVM: security.capability
[    0.105694] pinctrl core: initialized pinctrl subsystem
[    0.105694] xen:grant_table: Grant tables using version 2 layout
[    0.105694] Grant table initialized
[    0.105694] regulator-dummy: no parameters
[    0.126093] RTC time: 165:165:165, date: 165/165/65
[    0.126131] NET: Registered protocol family 16
[    0.126194] PCI: setting up Xen PCI frontend stub
[    0.126194] bio: create slab <bio-0> at 0
[    0.128055] ACPI: Interpreter disabled.
[    0.128065] xen:balloon: Initialising balloon driver
[    0.128137] xen_balloon: Initialising balloon driver
[    0.128137] vgaarb: loaded
[    0.128148] SCSI subsystem initialized
[    0.128175] usbcore: registered new interface driver usbfs
[    0.128175] usbcore: registered new interface driver hub
[    0.128175] usbcore: registered new device driver usb
[    0.128175] PCI: System does not support PCI
[    0.128175] PCI: System does not support PCI
[    0.128175] NetLabel: Initializing
[    0.128175] NetLabel:  domain hash size = 128
[    0.128175] NetLabel:  protocols = UNLABELED CIPSOv4
[    0.128175] NetLabel:  unlabeled traffic allowed by default
[    0.128175] Switched to clocksource xen
[    0.132453] AppArmor: AppArmor Filesystem Enabled
[    0.132464] pnp: PnP ACPI: disabled
[    0.134454] NET: Registered protocol family 2
[    0.134748] TCP established hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.135024] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    0.174348] TCP: Hash tables configured (established 131072 bind 65536)
[    0.174397] TCP: reno registered
[    0.174433] UDP hash table entries: 8192 (order: 6, 262144 bytes)
[    0.174503] UDP-Lite hash table entries: 8192 (order: 6, 262144 bytes)
[    0.174609] NET: Registered protocol family 1
[    0.174657] Trying to unpack rootfs image as initramfs...
[    0.190047] Freeing initrd memory: 16492K (ffff880002404000 - ffff88000341f000)
[    0.190140] platform rtc_cmos: registered platform RTC device (no PNP device found)
[    0.190348] microcode: CPU0 sig=0x306e4, pf=0x1, revision=0x428
[    0.190357] microcode: CPU1 sig=0x306e4, pf=0x1, revision=0x428
[    0.190369] microcode: CPU2 sig=0x306e4, pf=0x1, revision=0x428
[    0.190382] microcode: CPU3 sig=0x306e4, pf=0x1, revision=0x428
[    0.190394] microcode: CPU4 sig=0x306e4, pf=0x1, revision=0x428
[    0.190406] microcode: CPU5 sig=0x306e4, pf=0x1, revision=0x428
[    0.190419] microcode: CPU6 sig=0x306e4, pf=0x1, revision=0x428
[    0.190431] microcode: CPU7 sig=0x306e4, pf=0x1, revision=0x428
[    0.190502] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
[    0.190507] Scanning for low memory corruption every 60 seconds
[    0.190779] Initialise system trusted keyring
[    0.190819] audit: initializing netlink socket (disabled)
[    0.190831] type=2000 audit(1488951699.405:1): initialized
[    0.211597] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    0.212286] zbud: loaded
[    0.212424] VFS: Disk quotas dquot_6.5.2
[    0.212464] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.212837] fuse init (API version 7.22)
[    0.212902] msgmni has been set to 30079
[    0.212955] Key type big_key registered
[    0.213493] Key type asymmetric registered
[    0.213497] Asymmetric key parser 'x509' registered
[    0.213520] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
[    0.213579] io scheduler noop registered
[    0.213583] io scheduler deadline registered (default)
[    0.213605] io scheduler cfq registered
[    0.213647] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    0.213658] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[    0.213697] ipmi message handler version 39.2
[    0.214065] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
[    0.215290] Linux agpgart interface v0.103
[    0.216417] brd: module loaded
[    0.217033] loop: module loaded
[    0.218243] libphy: Fixed MDIO Bus: probed
[    0.218303] tun: Universal TUN/TAP device driver, 1.6
[    0.218306] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[    0.218345] PPP generic driver version 2.4.2
[    0.218383] xen_netfront: Initialising Xen virtual ethernet driver
[    0.218701] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.218711] ehci-pci: EHCI PCI platform driver
[    0.218721] ehci-platform: EHCI generic platform driver
[    0.218728] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.218730] ohci-pci: OHCI PCI platform driver
[    0.218740] ohci-platform: OHCI generic platform driver
[    0.218747] uhci_hcd: USB Universal Host Controller Interface driver
[    0.218792] i8042: PNP: No PS/2 controller found. Probing ports directly.
[    0.219084] blkfront: xvda1: barrier or flush: disabled; persistent grants: disabled; indirect descriptors: enabled;
[    1.219180] i8042: No controller found
[    1.219319] mousedev: PS/2 mouse device common for all mice
[    1.279012] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[    1.279064] rtc_cmos: probe of rtc_cmos failed with error -38
[    1.279140] device-mapper: uevent: version 1.0.3
[    1.279208] device-mapper: ioctl: 4.27.0-ioctl (2013-10-30) initialised: dm-devel@redhat.com
[    1.279216] ledtrig-cpu: registered to indicate activity on CPUs
[    1.279329] TCP: cubic registered
[    1.279406] NET: Registered protocol family 10
[    1.279640] NET: Registered protocol family 17
[    1.279651] Key type dns_resolver registered
[    1.279782] Loading compiled-in X.509 certificates
[    1.280457] Loaded X.509 cert 'Magrathea: Glacier signing key: a55b8de68a607adbfa6df96000506950cdaea041'
[    1.280473] registered taskstats version 1
[    1.283661] Key type trusted registered
[    1.289255] Key type encrypted registered
[    1.289261] AppArmor: AppArmor sha1 policy hashing enabled
[    1.289264] IMA: No TPM chip found, activating TPM-bypass!
[    1.289312] regulator-dummy: incomplete constraints, leaving on
[    1.289370]   Magic number: 1:252:3141
[    1.289426] /build/buildd/linux-3.13.0/drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
[    1.289484] BIOS EDD facility v0.16 2004-Jun-25, 0 devices found
[    1.289488] EDD information not available.
[    1.290120] Freeing unused kernel memory: 1336K (ffffffff81d20000 - ffffffff81e6e000)
[    1.290125] Write protecting the kernel read-only data: 12288k
[    1.293103] Freeing unused kernel memory: 788K (ffff88000173b000 - ffff880001800000)
[    1.293430] Freeing unused kernel memory: 688K (ffff880001b54000 - ffff880001c00000)
Loading, please wait...
[    1.317693] systemd-udevd[145]: starting version 204
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
[    1.415936] EXT4-fs (xvda1): mounted filesystem with ordered data mode. Opts: (null)
Begin: Running /scripts/local-bottom ... done.
done.
Begin: Running /scripts/init-bottom ... done.
[    1.594964] random: init urandom read with 79 bits of entropy available
[    1.643808] init: plymouth-upstart-bridge main process (201) terminated with status 1
[    1.643829] init: plymouth-upstart-bridge main process ended, respawning
[    1.651851] init: plymouth-upstart-bridge main process (208) terminated with status 1
[    1.651872] init: plymouth-upstart-bridge main process ended, respawning
[    1.656230] init: plymouth-upstart-bridge main process (214) terminated with status 1
[    1.656249] init: plymouth-upstart-bridge main process ended, respawning
[    1.662398] init: plymouth-upstart-bridge main process (216) terminated with status 1
[    1.662416] init: plymouth-upstart-bridge main process ended, respawning
[    1.668825] init: plymouth-upstart-bridge main process (219) terminated with status 1
[    1.668843] init: plymouth-upstart-bridge main process ended, respawning
[    1.773223] EXT4-fs (xvda1): warning: checktime reached, running e2fsck is recommended
[    1.773986] EXT4-fs (xvda1): re-mounted. Opts: (null)
[    1.911974] random: nonblocking pool is initialized
 * Stopping Send an event to indicate plymouth is up[74G[ OK ]

 * Starting Mount filesystems on boot[74G[ OK ]

 * Starting Populate /dev filesystem[74G[ OK ]

 * Starting Populate and link to /run filesystem[74G[ OK ]

 * Stopping Populate /dev filesystem[74G[ OK ]

 * Stopping Populate and link to /run filesystem[74G[ OK ]

 * Stopping Track if upstart is running in a container[74G[ OK ]

 * Starting Signal sysvinit that the rootfs is mounted[74G[ OK ]

 * Starting Initialize or finalize resolvconf[74G[ OK ]

 * Starting Clean /tmp directory[74G[ OK ]

 * Stopping Clean /tmp directory[74G[ OK ]

Cloud-init v. 0.7.5 running 'init-local' at Wed, 08 Mar 2017 05:41:41 +0000. Up 2.30 seconds.

cloud-init-nonet[2.56]: waiting 10 seconds for network device

 * Starting set console keymap[74G[ OK ]

 * Starting Signal sysvinit that virtual filesystems are mounted[74G[ OK ]

 * Starting Signal sysvinit that virtual filesystems are mounted[74G[ OK ]

 * Starting Bridge udev events into upstart[74G[ OK ]

 * Starting Signal sysvinit that remote filesystems are mounted[74G[ OK ]

 * Starting device node and kernel event manager[74G[ OK ]

 * Stopping set console keymap[74G[ OK ]

 * Starting load modules from /etc/modules[74G[ OK ]

 * Stopping cold plug devices[74G[[31mfail[39;49m]

 * Starting log initial device creation[74G[ OK ]

 * Stopping log initial device creation[74G[ OK ]

 * Stopping load modules from /etc/modules[74G[ OK ]

 * Starting load fallback graphics devices[74G[ OK ]

 * Stopping load fallback graphics devices[74G[ OK ]

 * Starting set console font[74G[ OK ]

 * Stopping set console font[74G[ OK ]

 * Starting userspace bootsplash[74G[ OK ]

 * Starting Send an event to indicate plymouth is up[74G[ OK ]

 * Stopping userspace bootsplash[74G[ OK ]

 * Stopping Send an event to indicate plymouth is up[74G[ OK ]

The disk drive for /data is not ready yet or not present.

keys:Continue to wait, or Press S to skip mounting or M for manual recovery

cloud-init-nonet[12.56]: waiting 120 seconds for network device

cloud-init-nonet[132.57]: gave up waiting for a network device.

Cloud-init v. 0.7.5 running 'init' at Wed, 08 Mar 2017 05:43:51 +0000. Up 132.75 seconds.

ci-info: ++++++++++++++++++++Net device info++++++++++++++++++++

ci-info: +--------+-------+---------+------+-------------------+

ci-info: | Device |   Up  | Address | Mask |     Hw-Address    |

ci-info: +--------+-------+---------+------+-------------------+

ci-info: |   lo   | False |    .    |  .   |         .         |

ci-info: |  eth0  | False |    .    |  .   | 12:11:50:ff:66:ed |

ci-info: +--------+-------+---------+------+-------------------+

ci-info: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Route info failed!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

2017-03-08 05:43:52,140 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [0/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:43:53,143 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [1/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:43:54,146 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [2/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:43:55,149 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [3/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:43:56,152 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [4/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:43:57,154 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [5/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:43:59,158 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [7/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:44:01,162 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [9/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:44:03,166 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [11/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:44:05,170 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [13/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:44:07,174 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [15/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:44:10,178 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [18/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:44:13,183 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [21/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:44:16,188 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [24/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:44:19,193 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [27/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:44:22,198 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [30/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:44:26,204 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [34/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:44:30,210 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [38/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

2017-03-08 05:44:34,215 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [42/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by <class 'socket.error'>: [Errno 101] Network is unreachable)]

Close

I looked at some of the solutions on the internet but all of them talk about fiddling with /etc/sysconfig/network-scripts etc, which does not exist on ubuntu. I went to /etc/network/interfaces.d/50-cloud-init (by attaching this volume to another running instance) but i didnt have any luck in fixing the problem. What should i do?

Thanks


Solution

  • If anybody has a similar problem: the ubuntu 14.04 is screwed and becomes brain dead if you restart it as it has DHCP problems. AWS support will ask you create a new instance and attach the volume (your initial volume which does not boot) and try to increase the timeout (it would be 300 by default). THIS DOES NOT HELP.

    The ONLY way to make this work is to create a new instance with ubuntu 16.04 (or wtv is the latest/greatest) and attach your problematic volume as the secondary mount. Its a pain but AWS has to be blamed for giving users ubuntu 14.04 as an OS option when clearly it has problems.