Installing USB driver for Nexus 4 (with KitKat) on Windows 8 64-bit - "no compatible software driver"

I'm desperately trying to install ADB USB drivers for my Nexus 4, which (as you might know) are required for any form of Android Development. You could also say, that they are the single most necessary tool used for Android programming, right? I have already often set up an Android Development Environment for my old HTC Desire, but I am failing with setting it up for my Nexus 4.

The way I go about it:

• Running Windows 8 and 8.1 Preview (tried everything on two different machines)
• Downloading the Android Development Tools from Google (SDK+ADT+Eclipse+etc.)
• Downloading the Google USB Drivers (v8) via Android SDK Manager
• In the Device Manager i have an entry "Nexus 4" which I right-click and then I select "Update Driver Software..."
• -> Browse my Computer for driver software -> Let me pick a list of device drivers on my computer -> Have Disk... -> Choose Path to Google Drivers -> Dialog occurs:
• "The folder you specified doesn't contain a compatible software driver for your device. [...]"

I really do not know what the problem is. Some things I already tried include:

• Doing everything on USB3/USB2 Port
• Changing Computer Connection Mode to PTP (instead of MTP)
• Doing everything on Win8/Win8.1 Preview
• I tried to install the universal ADB Driver: download.clockworkmod.com/test/UniversalAdbDriverSetup6.msi without any significant result

I fear that the problem is because of the new driver version (v8) or KitKat, because I can't find any working solutions on the net and KitKat is relatively new, however, does anyone have a solution to this?

Enable Debug-Mode on the device.

Steps to enable debug mode:

Before you do anything in order to develop on a Nexus 4 (at least in my experience), do the following:

• Open Settings App
• Navigate to About Phone
• click Build number seven times to active the Developer Mode - Settings main screen->Developer options->enable "USB debugging"

In my defense: It doesn't say anywhere, that this is a prequisite for Android development. I knew of this option, but since the last time I set this option on a device is years ago, I just thought to myself "well... I do not want to debug, so this option is irrelevant"