Changing event start date from "date" to "dateTime"

I am trying to patch an existing event with new start and end date using the Google API Explorer

So my event at the moment looks like this

{
 "kind": "calendar#event",
 "etag": "\"2912997881756000\"",
 "id": "3fpkrr85sdfdgsdfsdsdflgn7vk74qhiv2o",
 "status": "confirmed",
 "htmlLink": "https://www.google.com/calendar/event?eid=M2Zwa3JsdfsdfyODVudWZobGduN3ZrNzRxaGl2Mm8gZHlsbsdfsdfi5pb19xNjUwcWRhcnYyam9vYWYzcTdudmhpc2ZvNEBn",
 "created": "2016-02-26T14:32:33.000Z",
 "updated": "2016-02-26T15:02:20.878Z",
 "summary": "aaaa",
 "creator": {
  "email": "xxxx@yyy.com",
  "displayName": "name"
 },
 "organizer": {
  "email": "xxxxxx@group.calendar.google.com",
  "displayName": "Display",
  "self": true
 },
 "start": {
  "date": "2016-02-26"
 },
 "end": {
  "date": "2016-03-02"
 },
 "iCalUID": "3fpkrr85nufasdfsdfsadfasdfhlgn7vk74qhiv2o@google.com",
 "sequence": 2,

 "reminders": {
  "useDefault": true
 }
}

And the request I am making from the Google API Explorer is something like this

PATCH https://www.googleapis.com/calendar/v3/calendars/ddfdfdfdfdf%40group.calendar.google.com/events/3fpkrr85nufhlgn7sdfsafdfdsghgffdhvk74qhiv2o?fields=start&key={YOUR_API_KEY}

{
 "start": {
  "dateTime": "2016-02-16T12:00:00+01:00"
 },
 "end": {
  "dateTime": "2016-02-18T13:00:00+01:00"
 }
}

But I always get the following error

{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "invalid",
    "message": "Invalid start time."
   }
  ],
  "code": 400,
  "message": "Invalid start time."
 }
}

I am guessing this is happening because the event is an All Day event which means start is a Date field and I am sending DateTime but this is exactly what I want to do. Can't I just simply change the type of start and end from Date to DateTime?

UPDATE

Here is the php code for the path request

$client = $this->container->get('google.calendar.client');
                $client->setAccessToken($this->auth()->getIdentity()->getGoogleAccessToken());
                $service = new \Google_Service_Calendar($client);
                $event = new \Google_Service_Calendar_Event($eventData);
                $event = $service->events->patch($this->auth()->getIdentity()->getGoogleCalendarId(), $item->getGoogleId(), $event);
                $item->setGoogleId($event->getId());
                $this->getItemRepo()->save($item);

There seems to be a problem with leftover dates. Setting dates explicitly to null works:

{
 "start": {
  "dateTime": "2016-02-16T12:00:00+01:00",
  "date": null
 },
 "end": {
  "dateTime": "2016-02-18T13:00:00+01:00",
  "date": null
 }
}