pythoncolorsphilips-huembedtls

Python - XY colour values not sending correctly using Philips Hue Entertainment API (via DTLS/PSK)


Edited to add link to github repository with the full class:

https://github.com/EvillerBobUK/pyHue-BridgeLink-Example

Requires you to have the Philips Hue hardware and various DTSL/PSK packages for the examples to work


I'm writing a small program in Python 3, running in Ubuntu on a Raspberry Pi, to send instructions to smart lights via a Philips Hue bridge (version 2).

The program works in most modes except one (detailed below) so I believe the connection itself is functioning properly. I think the problem is in the construction of the datagram for a specific mode - possibly to do with the Bridge only using 12 bit resolution for X and Y floats in that mode according to the docs? - but this is an area I'm really not familiar with and I just can't see where I'm going wrong!

The PH API has a "standard" mode where instructions are sent via https, and an "entertainment" mode where instructions are broadcast via UDP using dtls/psk. Entertainment mode supports sending data as RGB or XY + Brightness, with either version equating to 3 x 16bit datatypes. Endianess is given in the API guide as "Big-Endian (network)"; both BigEndian and network have been tried and give the same results.

RGB is sent as three integers between 0-254.

XY is sent as two floats between 0.0-1.0. The API documentation seems to suggest Brightness should be send as an integer between 0-254 which doesn't work. Experimentally, sending it as a float between 0.0-1.0 works as expected.

I'm using a snippet of code I grabbed off the internet somewhere to create the DTS/PSK connection and the packed struct (I really was struggling with the whole DTLS/PSK/Datagram thing), though it was only being used for RGB in the code if I recall. From my understanding of the Philips Hue API guides the struct is built and sent in the same way, just changing the colourspace flag and the datatype which I've done, so in theory it should work?

If I send the instruction {"xy":[0.6915,0.3083],"bri":254} via https using the standard mode, the light turns red as expected.

If I send the instruction [(3, 255, 0, 0)],'RGB' via UDP using the entertainment mode, the light turns red as expected and the Philips Hue API Debug CLIP tool reports "xy": [0.6915,0.3083] as it should.

If I send the instruction [(3, 0.6915,0.3083, 0.1)],'XYB' via UDP using the entertainment mode, the light turns a very whitish-blue and the Philips Hue API Debug tool reports "xy": [0.2245,0.2065] instead!

Playing around with different values between 0.0-1.0 always seem to end with some kind of pale blue. At one point I ran a loop which just pushed the numbers in all directions and out of expected range, which did result in occasionally getting yellows, greens, and rarely reds, but usually when the values being sent where massively outside of range/negative values. None of those late night desperate tests were properly documented or pointed to any kind of pattern/logic that I could see in the results!

Any help in understanding the issue and resolving it would be greatly, greatly appreciated!

CODE EXTRACT FROM CLASS

class pyHue_BridgeLink:
    def __init__(self, bridgename=False,config=False):
        # self.broadcast is a class-level variable for holding the datagram packed struct.      
        self.broadcast = None

    # put is the function that takes the instructions and turns them into a https put request, 
    # which is then sent to the bridge for forwarding to the appropriate light.
    def put(self,url,request,payload,sslverify=False):
        return requests.put(f"{url}{request}", json=payload, verify=sslverify).json() 

    # prepare_broadcast is the function that takes the instructions and turns them into a datagram,
    # which is then sent to the bridge (by a call elsewhere to send_broadcast) for broadcasting to the
    # entertainment light group.
    # Datatype 'e' is, I believe, a 16bit float.
    # Endian type > (BigEndian) and ! (network) have both been tried
    def prepare_broadcast(self, states, colourspace='RGB'): #colourspace = 'RGB' or 'XYB'
        if colourspace == 'XYB':
            cs = 0x01
            datatypes = ">BHeee"
        else:
            cs = 0x00
            datatypes = ">BHHHH"
        count = len(states)
        self.broadcast = bytearray([0]*(16+count*9))
        struct.pack_into(">9s2BB2BBB", self.broadcast, 0,
                         "HueStream".encode('ascii'),  # Protocol Name (fixed)
                         0x01, 0x00,                   # Version (=01.00)
                         0x00,                         # Sequence ID (ignored)
                         0x00, 0x00,                   # Reserved (zeros)
                         cs,                           # Color Space (RGB=0x00, XYB=0x01)
                         0x00                          # Reserved (zero)
                         )
        for i in range(count):  # Step through each set of instructions in "states"
            struct.pack_into(datatypes, self.broadcast, 16 + i*9,
                             0x00,                      # Type: Light
                             states[i][0],              # Light ID
                             states[i][1],              # Red/X
                             states[i][2],              # Blue/Y
                             states[i][3]               # Green/Brightness
                             )

    # send_broadcast passes the prepared datagram held in self.broadcast to the socket
    # for sending via DTLS/PSK which is, as far as I can establish, some kind of witchcraft.
    def send_broadcast(self):
        if self.sock:
            self.sock.send(self.broadcast) 

    # prepare_and_send_broadcast is an intermediary function that, unsurprisingly, calls the
    # prepare_broadcast command above then calls the send_broadcast command. There are cases where the
    # self.broadcast packed struct will be sent at a different time to when it was prepared. This
    # function is used when both need to happen at the same time.
    def prepare_and_send_broadcast(self, states, colourspace='RGB'):
        self.prepare_broadcast(states, colourspace)
        self.send_broadcast()

EXAMPLE USAGE

bl = pyHue_BridgeLink("BridgeOne")

# Using the standard API to send via https. This works correctly!
# Physical light turns red as expected.
# Philips Hue API Debug CLIP tool reports "xy": [0.6915,0.3083] as it should.
bl.put(bl.url,'lights/3/state',{"xy":[0.6915,0.3083],"bri":254})

# Switching to Entertainment mode...
bl.enable_streaming()

# Using the entertainment API to stream RGB instructions via UDP. This works correctly!
# Physical light turns red as expected.
# Philips Hue API Debug CLIP tool reports "xy": [0.6915,0.3083] as it should.
bl.prepare_and_send_broadcast([(3, 255,0,0)],'RGB')

# Using the entertainment API to stream XY+Brightness instructions via UDP. This fails!
# Physical light turns a very whitish-blue.
# Philips Hue API Debug CLIP tool reports "xy": [0.2245,0.2065] which is very wrong.
bl.prepare_and_send_broadcast([(3, 0.6915,0.3083, 0.1)],'XYB')

# Switching back out of Entertainment mode...
bl.disable_streaming()

Edited to add link to github repository with the full class:

https://github.com/EvillerBobUK/pyHue-BridgeLink-Example

Requires you to have the Philips Hue hardware and various DTSL/PSK packages for the examples to work


Solution

  • Ran into the same issue and found the fix. The key is where the Hue docs state

    For xy, the minimum value of 0x0000 is equivalent to 0.00000, and 0xffff is equivalent to 1.00000

    So to get the proper hex value you need something like the following to get the correct values for each of the two bytes (where x and y are the decimal values like "0.6915" / "0.3083"):

    x_1, x_2 = int(x * 65535).to_bytes(2, 'big')
    y_1, y_2 = int(y * 65535).to_bytes(2, 'big')
    

    then in the struct.pack_into piece just reference them as individual bytes (I'm using the V2 of the Entertainment API format but it's similar to the code above):

    struct.pack_into(">9s2BB2BBB36sB6BB6B", light_data, 0,
                         "HueStream".encode('ascii'),  # Protocol Name (fixed)
                         0x02, 0x00,                   # Version (=02.00)
                         0x00,                         # Sequence ID (ignored)
                         0x00, 0x00,                   # Reserved (zeros)
                         0x01,                           # Color Space (RGB=0x00, XYB=0x01)
                         0x00,                          # Reserved (zero)
                         entertainment_id.encode('ascii'), # Entertainment id
                         0x00,                          # Channel 0
                         x_0, x_1, y_0, y_1, b_0, b_1,  # Channel 0 values
                         0x01,                          # Channel 1
                         x_0, x_1, y_0, y_1, b_0, b_1   # Channel 1 values
                         )